Commit Graph

830 Commits

Author SHA1 Message Date
Louis King d509f82aec docs: document Route Health Monitoring for v0.16 release
- Add docs/routes.md feature/operational doc (evaluation semantics,
  visibility, observer scoping, background evaluator, prerequisites)
- Add v0.16.0 'Route Health Monitoring' section to docs/upgrading.md
  (above the existing Observer Ingestion Filters subsection)
- docs/configuration.md: cross-ref header + FEATURE_ROUTES (Feature Flags)
  and ROUTE_EVALUATOR_INTERVAL_SECONDS (Collector) rows
- README.md: Features bullet + docs tree entry + Documentation list entry
- docs/seeding.md: cross-link to routes.md from the Routes section
- docker-compose.yml: wire FEATURE_ROUTES (web) and
  ROUTE_EVALUATOR_INTERVAL_SECONDS (collector) as passthroughs
2026-07-17 23:38:45 +01:00
Louis King 4a2add26a4 fix: satisfy mypy on stats["errors"] union type in test_cli.py 2026-07-16 23:34:45 +01:00
Louis King a64d0b5561 test: close Route Health Monitoring coverage gaps (PR #308)
Add 49 targeted tests across 8 files to lift patch coverage from 76.9%
to ~95%+:

- collector/cli.py (3% -> covered): TestImportRoutes (12 tests for
  _import_routes: happy-path, upsert, all error branches, exception
  handler) + TestRouteSeedImport (seed command integration).
- api/routes/routes.py: update_route all-field branch, duplicate-label
  409, observer sync, unresolved-nodes 400, hidden-route 404, get_route
  contributing-observers, create with observers, preview guards,
  history retention clamp.
- collector/routes.py: evaluate_all_routes exception handler, preview
  early-return guards, evaluate_route_history < 2 expected, limit/until/
  observer params on fetch_candidate_paths, _route_expected_hashes
  derive branch, _has_any_hops_per_day empty guard.
- collector/subscriber.py: TestRouteEvaluatorScheduler (disabled,
  runs-and-stops, error-is-logged) mirroring TestSpamRescoreScheduler.
- common/schemas/routes.py: RouteUpdate + RoutePreviewRequest validators
  (min nodes, distinct, clear_threshold).
- collector/route_evaluator.py: evaluation exception handler.
- common/config.py: route_evaluator_interval_seconds default + custom.
- api/routes/packet_groups.py: mixed-visibility representative selection.
2026-07-16 23:30:28 +01:00
Louis King 8ce40fc75e Merge branch 'main' of github.com:ipnet-mesh/meshcore-hub into feat/route-health-monitoring 2026-07-16 23:08:22 +01:00
JingleManSweep b5bce562a5 Merge pull request #307 from ipnet-mesh/renovate/anomalyco-opencode-1.x
chore(deps): update anomalyco/opencode action to v1.18.3
2026-07-16 23:07:38 +01:00
JingleManSweep 2031e960d7 Merge branch 'main' into renovate/anomalyco-opencode-1.x 2026-07-16 23:05:38 +01:00
JingleManSweep e465cd34ea Merge pull request #309 from ipnet-mesh/renovate/tailwindcss-monorepo
chore(deps): update tailwindcss monorepo to v4.3.3
2026-07-16 23:05:19 +01:00
JingleManSweep ac73b35e92 Merge branch 'main' into renovate/tailwindcss-monorepo 2026-07-16 23:02:26 +01:00
JingleManSweep f2f93656f5 Merge branch 'main' into renovate/anomalyco-opencode-1.x 2026-07-16 23:00:40 +01:00
JingleManSweep 6b823027ef Merge pull request #305 from ipnet-mesh/renovate/daisyui-5.x-lockfile
chore(deps): update dependency daisyui to v5.6.18
2026-07-16 23:00:21 +01:00
renovate[bot] 346d75415a chore(deps): update anomalyco/opencode action to v1.18.3 2026-07-16 21:59:45 +00:00
renovate[bot] 1f6c144322 chore(deps): update tailwindcss monorepo to v4.3.3 2026-07-16 21:59:40 +00:00
Louis King b8b7df0966 ui(routes): split from/to header into aligned icon grid
The route card header wrapped onto two lines when from/to labels were
long. Replace the inline "from -> to" title with a two-row CSS grid
(auto | 1fr) so labels align vertically, each preceded by a dedicated
SVG icon (iconRouteFrom/iconRouteTo: arrow departing/arriving at a
dot). Long labels now ellipsize with a title tooltip.

Move the reversibility indicator (reversible <-> vs one-way ->) out of
the title into a ghost badge beside the quality badge, where it no
longer fights the grid layout.

Also nudge spacing between "Recent Packets" and its path rows (mt-1 ->
mt-2) for a little breathing room.
2026-07-16 22:56:48 +01:00
Louis King 30f18561bf refactor: consolidate route health migrations into one clean migration
Squashes four formerly-separate tail migrations into a single migration
(ec40c67c8c83) that builds the final schema directly off 57bb65130b97:

- 8f2a3c4d5e6f route health tables + packet_path_hops backfill
- c0d1e2f3a4b5 deduplicate nodes by public_key (idempotent data fix)
- 76513f4c57e9 ix_packet_path_hops_received_at index
- 71f6e01e4bf6 rename degraded_threshold -> clear_threshold

The rename runs as a no-op: routes.clear_threshold and
route_results.effective_clear are born with their final names. The
received_at index is created with the table, and the node dedup is
preserved verbatim (no-op on a clean DB).

Also drops the redundant uq_route_results_route_id unique constraint:
route_results.route_id now has a single unique index matching the model,
eliminating a pre-existing autogenerate drift.

Net schema is identical to running all four migrations. Verified with a
fresh alembic upgrade head + alembic check (no drift) on SQLite.
2026-07-16 22:34:13 +01:00
Louis King ad07c1521c feat: route health history, compact cards, clear_threshold rename
- Add per-route health history endpoint (GET /routes/{id}/history)
- Optimize history fetch with _fetch_matching_hops (SQL-level prefix filter)
- Add standalone received_at index on packet_path_hops
- Fix cache key collision by including URL path in _routes_key_builder
- Fix Chart.js canvas reuse with Chart.getChart() guard
- Remove fleet overview endpoint (502 timeout on large datasets)
- Redesign route cards: always expanded, compact icon stats row,
  per-segment date labels, loading spinner, bottom-pinned admin buttons
- Rename degraded_threshold -> clear_threshold across model, schema,
  API, metrics, CLI, frontend, tests, and seed config
- Add iconClock to icons.js
- Update AGENTS.md: random revision IDs, never build images
- Two Alembic migrations: received_at index, column rename
2026-07-15 23:37:03 +01:00
Louis King ae22dff4bc fix: inject now param into run_evaluation for deterministic tests
Test data used a hardcoded _NOW (2026-07-12 12:00 UTC) while
run_evaluation used real datetime.now(), causing a time-window flake
once wall-clock time exceeded _NOW + window_hours (24h). Tests now
pass now=_NOW explicitly; production callers are unaffected (defaults
to datetime.now(timezone.utc) when not provided).
2026-07-13 16:23:00 +01:00
Louis King 8a1b6d787a fix: filter observer autocomplete to is_observer nodes
The Route UI's observer search was hitting GET /api/v1/nodes without
the observer filter, showing all nodes instead of only observer nodes.
The backend already supports ?observer=true via an indexed column —
just needed to pass the param from handleObsSearch and handleObsKeydown.
2026-07-13 16:08:51 +01:00
Louis King 1e87b7abb5 Merge branch 'main' of github.com:ipnet-mesh/meshcore-hub into docs/mesh-link-monitoring-plan 2026-07-13 16:05:14 +01:00
Louis King d10f2b4c9f fix: deduplicate nodes by public_key and re-assert uniqueness
Production DB restore bypassed UNIQUE enforcement on nodes.public_key,
leaving duplicate rows that caused MultipleResultsFound in node lookups
(GET /api/v1/nodes/{public_key} and collector find-or-create handlers).

Migration merges duplicates (winner = earliest first_seen), re-points all
14 FK columns referencing nodes.id, merges scalar fields, and re-creates
the UNIQUE index. Superset of b1c2d3e4f5a6 which only covered 9 FKs.
2026-07-13 15:49:25 +01:00
Louis King 1ee01a65fe refactor: replace route name with from/to endpoint labels
Route identity is now a (from_label, to_label) composite unique pair
instead of a single name column.  The three route migrations (create
tables, add reversible, replace name) are collapsed into one clean
migration since the feature hasn't shipped yet — 8f2a3c4d5e6f creates
all five tables with from_label/to_label + reversible from the start,
no intermediate steps.

Engine: recent_matches now returns the sliced subpath between the
matched endpoints, not the full path.  Diagnosis text moved to a hover
tooltip on the quality badge.  Cards sorted by from_label.  Seed YAML
switched to a list format with from/to keys; upsert by (from_label,
to_label).  Plan/tasks docs updated to present the final schema.
2026-07-13 13:35:58 +01:00
Louis King 6a4329b9d2 fix: path highlight lookup should use prefix match, not exact
Routes with match_width smaller than the mesh protocol's path hash width
would never highlight matched nodes in recent matches.  The collector's
is_subsequence uses startswith() (prefix match), but the JS lookup used
Map.get() (exact match).  Truncate node_hash to 2*match_width chars
before lookup so 'a1b2c3' matches expected_hash 'a1'.
2026-07-12 23:03:04 +01:00
Louis King 27d6160164 fix: use boolean literal for reversible server_default
PostgreSQL rejects DEFAULT 1 for boolean columns; use DEFAULT true
which works on both PostgreSQL and SQLite (3.23.0+).
2026-07-12 22:46:35 +01:00
Louis King 818d4a2abe fix: collision in reversible migration revision ID
The reversible migration used revision ID a1b2c3d4e5f6, which was
already taken by the rename_receiver_to_observer migration.  Renamed
to f1e2d3c4b5a6.
2026-07-12 22:43:39 +01:00
Louis King 0e202b3999 feat: reversible route matching
Routes now default to bidirectional matching — packets traversing the
configured path in reverse (C->B->A instead of A->B->C) also count
towards health.  A 'reversible' toggle lets users constrain a route
to one direction when needed.

- Route model: new 'reversible' column (default true, server_default 1)
- Migration: a1b2c3d4e5f6 (additive ALTER TABLE)
- Matching engine: _match_hops + _fetch_candidate_paths_maybe_bidirectional
  helpers shared across evaluate_route, recent_matches, and preview_route
- API schemas: reversible on RouteCreate/Update/Read/Detail/PreviewRequest
- SPA: toggle in modal; card path chips show <-> for reversible, -> otherwise
- Seed loader + example YAML: reads 'reversible' key
- 4 new collector tests (reverse match, non-reversible, both-directions, API)
2026-07-12 22:39:15 +01:00
Louis King 14fbc45387 feat: route health monitoring with visual path builder
Add complete route health monitoring feature that tracks whether packets
traverse expected multi-hop paths through the mesh network.

Models & migration:
- 5 new models: PacketPathHop, Route, RouteNode, RouteObserver, RouteResult
- Alembic migration with keyset-paginated backfill of existing packets

Collector:
- store_raw_packet refactored to persist path hops via bulk insert
- Matching engine (collector/routes.py) with subsequence matching, quality
  bands (clear/marginal/failing/no_coverage), and collision detection
- Background route evaluator (60s loop) wired into subscriber lifespan
- Route seed loader in CLI (resolves by public_key, matching YAML format)

API:
- 6 CRUD endpoints + preview endpoint under /api/v1/routes
- Schemas accept node_public_keys (64-char hex) instead of internal UUIDs
- 5 Prometheus gauges for route health metrics
- Packet groups endpoint reads from hop table

Web UI:
- Full SPA routes page with summary strip, grouped cards, expandable detail
- Routes nav entry in both desktop (spa.html) and mobile (app.js) navbars
- Home page nav card with feature gate
- API proxy access mapping for v1/routes endpoints
- Visual node-search path builder with autocomplete dropdown, ordered chips
  with reorder/remove controls, and paste-64-char-key support
- Observer picker with same search UX (unordered chips)
- i18n strings in en.json and nl.json

Config:
- feature_routes flag (default: true)
- route_evaluator_interval_seconds (default: 60)
- routes_file seed path
- example/seed/routes.yaml
2026-07-12 22:26:44 +01:00
renovate[bot] e60dc7ba26 chore(deps): update dependency daisyui to v5.6.18 2026-07-12 17:47:38 +00:00
JingleManSweep 2455d5af0e Merge pull request #304 from ipnet-mesh/jinglemansweep-patch-1
Add GitHub funding link to FUNDING.yml
2026-07-09 22:18:00 +01:00
JingleManSweep 91a65807ad Add GitHub funding link to FUNDING.yml 2026-07-09 22:16:08 +01:00
JingleManSweep f48c37578d Merge pull request #302 from ipnet-mesh/renovate/daisyui-5.x-lockfile
chore(deps): update dependency daisyui to v5.6.16
2026-07-09 21:47:32 +01:00
renovate[bot] bc8a33eb34 chore(deps): update dependency daisyui to v5.6.16 2026-07-09 20:02:53 +00:00
JingleManSweep 0b4621e7cc Merge pull request #303 from ipnet-mesh/renovate/anomalyco-opencode-1.x
chore(deps): update anomalyco/opencode action to v1.17.18
2026-07-09 21:02:01 +01:00
renovate[bot] 0981b2ec73 chore(deps): update anomalyco/opencode action to v1.17.18 2026-07-09 19:40:13 +00:00
JingleManSweep 9420164ed2 Merge pull request #301 from ipnet-mesh/feat/observer-area-filters
feat(web): group observer filter badges by area tag
v0.15.10
2026-07-07 22:21:03 +01:00
Louis King b1ca4fa1f3 Merge branch 'main' of github.com:ipnet-mesh/meshcore-hub into feat/observer-area-filters 2026-07-07 22:16:39 +01:00
JingleManSweep 69c0922b15 Merge pull request #299 from ipnet-mesh/renovate/anomalyco-opencode-1.x
chore(deps): update anomalyco/opencode action to v1.17.15
2026-07-07 22:16:21 +01:00
renovate[bot] 857e13c3fe chore(deps): update anomalyco/opencode action to v1.17.15 2026-07-07 21:14:13 +00:00
JingleManSweep c2cc05da8d Merge pull request #300 from ipnet-mesh/renovate/daisyui-5.x-lockfile
chore(deps): update dependency daisyui to v5.6.14
2026-07-07 22:13:41 +01:00
Louis King d1a3605a54 feat(web): group observer filter badges by area tag
Switch the Adverts and Messages observer filter from one badge per observer
node to one badge per unique area tag value (e.g. IP2, IP3, IP4). Toggling
an area badge enables/disables all observers in that area together.

Observers without an area tag are hidden from the filter row. Uses a new
localStorage key (meshcore-observer-areas-disabled) to avoid stale public
keys being misread as area codes. The old key is cleaned up on SPA boot.

Frontend-only: no backend, API, schema, or migration changes.
2026-07-07 22:13:15 +01:00
Louis King d60e2e8b1d docs: review link plan, add UI design, perf decisions, packet-detail swap
Review pass: fix stale citations (event_observer upsert path/method,
raw_packet hop-insert point, web path prefixes, VISIBILITY_LEVELS
location, features dict range); add Review section.

UI Design: status-board layout, four-state health badge
(healthy/unhealthy/no_coverage/disabled), inline accordion expand,
segmented match_width control, lazy GET /{id} detail, mesh_links.*
i18n namespace, POST /links/preview endpoint.

Performance: fetch-and-check engine over N-way self-join,
(node_hash, received_at) + (raw_packet_id, position) indexes,
preview candidate cap + debounce, evaluator threshold short-circuit.

Phase 9: consolidate packet-detail path read onto packet_path_hops;
delete duplicate _extract_path_hashes. Enabled Future Capabilities note
(path-node filtering, relay stats).
2026-07-07 00:07:46 +01:00
Louis King ae832fe73f docs: add seeding support and default visibility to link plan
Add F8/T9 + Phase 8 seed loader (links.yaml via existing meshcore-hub seed command, mirroring channels.yaml), referencing path nodes by public_key with expected_hash derived by the importer. Refine visibility to keep auth-scope but default to community (like channels).
2026-07-06 23:32:24 +01:00
renovate[bot] adcddc2ca4 chore(deps): update dependency daisyui to v5.6.14 2026-07-06 19:52:46 +00:00
Louis King 7530f8cc15 docs: add mesh link monitoring plan
Plan for a new Link entity that monitors whether packets traverse a configured ordered sequence of mesh nodes within a time window + count threshold, with background evaluation and Prometheus exposure. Includes shareable plain-language overview.
2026-07-05 23:22:01 +01:00
JingleManSweep 62d5b637f0 Merge pull request #298 from ipnet-mesh/fix/illegal-user-name-header
fix(web): sanitize X-User-Name header to prevent 502 on registration
v0.15.9
2026-07-04 20:10:54 +01:00
Louis King cbd12b6c25 Merge branch 'main' of github.com:ipnet-mesh/meshcore-hub into fix/illegal-user-name-header 2026-07-04 20:08:12 +01:00
Louis King 7b44049e6e fix(web): sanitize X-User-Name header to prevent 502 on registration
When an IdP name claim contains trailing whitespace (e.g. 'Matt '), every
proxied API request failed with 502 'Illegal header value' because httpx
enforces RFC 7230 which forbids leading/trailing OWS in header values.

Defense in depth:
- strip_userinfo() trims the IdP name at ingress (oidc.py)
- _sanitize_header_value() removes CTL/DEL chars at both header-injection
  sites (API proxy + auth-callback bootstrap) in app.py
- update_profile() trims user-supplied names in PUT handler

X-User-Name is informational only (seeds non-unique UserProfile.name);
identity/auth are keyed on X-User-Id and X-User-Roles, so sanitization is
safe.
2026-07-04 20:06:04 +01:00
JingleManSweep dc72657b0a Merge pull request #297 from ipnet-mesh/feat/packet-breakdown-charts
feat: add packet breakdown charts to dashboard
v0.15.8
2026-07-04 15:14:19 +01:00
Louis King 03006091e5 feat: add packet breakdown charts to dashboard
Add two horizontal 100% stacked-bar charts to the Dashboard, gated
behind the existing packets feature flag:

- Packet Types: top 6 event_type values + 'other' rollup
- Path Bytes: 1b/2b/3b path-hash width distribution (NULL excluded)

Driven by a single new cached endpoint
GET /api/v1/dashboard/packet-breakdown?days=7 returning pre-bucketed
counts. Frontend normalizes to percentages via a new
createStackedBarChart helper in charts.js with a 7-hue oklch palette.
2026-07-04 15:11:17 +01:00
JingleManSweep b608d9873b Merge pull request #296 from ipnet-mesh/chore/update-observer-docs
Updated Observer Documentation/Examples
2026-07-04 14:19:45 +01:00
Louis King a2cbbd6468 Updated observer docs 2026-07-04 14:17:02 +01:00
JingleManSweep 024114fc5b Merge pull request #294 from ipnet-mesh/renovate/daisyui-5.x-lockfile
chore(deps): update dependency daisyui to v5.6.13
v0.15.7
2026-07-04 10:50:20 +01:00