Convert every remaining lit-html page to React 19 + TypeScript and wire
them directly into the router, removing all LitBridge usage from App.tsx:
- Home, CustomPage, Profile, Members, Channels
- Advertisements, Messages, Routes, Nodes, NodeDetail
- Packets, PacketDetail, PacketGroupDetail, Dashboard, MapPage
Pages use the shared React infrastructure (apiGet<T>, useAutoRefresh,
usePageTitle, useFormatDateTime, Pagination, FilterForm, SortableTable,
NodeDisplay, ObserverBadges, RouteTypeBadge, JsonTree, StatCard, icons).
Charts/maps still call window.Chart / window.L / window.QRCode / charts.js
globals — these move to react-chartjs-2 / react-leaflet in Phase 3.
The old lit-html code in spa/ is intentionally kept as the spa.html
fallback (rendered only when the Vite bundle is absent) and is still
referenced by 5 web tests; it will be removed in Phase 4.
Added IconSatelliteDish, IconRuler, IconHopSpan, IconPathLength icons.
Verified: tsc --noEmit clean, npm run build (94 modules),
pytest tests/test_web/ (256 passed), pre-commit (passed).
Adds a new nullable per-route knob that caps the total number of hops
in a candidate packet's path. Packets whose path exceeds the cap are
dropped from matching consideration entirely (before the subsequence
matcher runs), so over-long paths never count toward
packet_count_threshold. Complements the existing max_hop_span, which
only constrains the gap between the first and last matched configured
node.
- Route model + migration (additive, nullable, default null = unlimited)
- Threaded through matcher chain (_subsequence_indices early-return)
- All 5 evaluate/preview/recent_matches call sites updated
- API serializer/create/update/preview passthrough
- CLI seed YAML import (update + create paths)
- Frontend: distinct icons for span (<-o->) vs path-length (|<->|),
always-rendered badges with infinity fallback, hover tooltips on
every stats row item, i18n keys (en + nl)
- Tests: matcher unit tests (within/exceeds cap), API round-trip,
CLI seed import
The route edit modal builds its PUT body with a ternary that collapses
an empty observer list to null:
observer_public_keys: observerPublicKeys.length > 0 ? observerPublicKeys : null,
Pydantic parses null as None, and the PUT handler's guard
if body.observer_public_keys is not None:
_sync_observers(session, route, observer_nodes)
skips the sync entirely when the field is None. So removing all
observers in the modal sent null -> no DB change. Adding observers
worked because a non-empty array passed the guard and _sync_observers
deleted + recreated.
The None-means-skip semantic is correct for true partial updates, so
the fix is on the frontend: the edit modal is a full-form PUT, so it
must always send the array. When empty, it sends [], which Pydantic
parses as [] (not None), the guard passes, and _sync_observers deletes
every existing RouteObserver row.
Tests: add test_update_clear_observers_with_empty_list next to the
existing test_update_observers as a regression guard. It seeds one
observer via PUT, then PUTs observer_public_keys: [] and asserts both
the response and a fresh GET come back with an empty list.
The Route Health and Routes Trend dashboard widgets pull from
GET /api/v1/dashboard/routes-overview, which previously filtered
routes by the caller's role tier (anonymous saw community, admin saw
all four tiers). Operators and admins therefore saw member/operator/
admin-tier routes mixed into the dashboard, even though the dedicated
/routes page is the working surface for managing those private tiers.
This change makes the dashboard widget surface ONLY community-tier
routes, regardless of the caller's role. Operators and admins still
see all four tiers on the /routes management page (unchanged).
Implementation:
- get_routes_overview: drop resolve_user_role/get_max_visibility_level;
push the filter into the SQL query
(where Route.visibility == RouteVisibility.COMMUNITY.value) so
higher-tier rows are no longer loaded just to be discarded.
- _dashboard_routes_overview_key_builder: drop the role dimension from
the cache key. The response is now identical across roles, so the
per-role cache slots were storing four identical copies. The prefix
is preserved so invalidate_routes' pattern invalidation still hits.
- Drop unused VISIBILITY_LEVELS import; add RouteVisibility to the
models import.
Tests:
- test_visibility_filter_hides_admin_routes renamed to
test_dashboard_only_shows_community_routes_regardless_of_role;
now seeds one route per visibility tier and asserts every role
(anonymous/member/operator/admin) sees only ['Public'].
- test_cache_key_is_role_scoped renamed to
test_cache_key_is_role_agnostic; asserts all four role-variants
produce the SAME cache key (no 'role=' dimension).
Change the new-route defaults so the create modal, REST API, YAML
import, and preview helper pre-fill packet_count_threshold=5 instead
of 3, and bump the effective-clear auto-multiplier from 2x to 3x so
the default comfort bar tracks to 15 for a default threshold of 5.
Schema/model/cli/preview:
- schemas/routes.py: RouteCreate + RoutePreviewRequest
packet_count_threshold default 3 -> 5 (clear_threshold stays None)
- models/route.py: Route INSERT default 3 -> 5 (clear_threshold stays
nullable, no default)
- collector/cli.py: YAML import fallbacks 3 -> 5
- collector/routes.py: preview helper fallback 3 -> 5; module constant
CLEAR_DEFAULT_MULTIPLIER 2 -> 3 (drives effective_clear_threshold)
- api/metrics.py: route_clear gauge help text 2x -> 3x
UI:
- web/spa/pages/routes.js: new-route modal pre-fills
packet_count_threshold=5; placeholder shows 3x threshold; parseInt
fallback 3 -> 5. clear_threshold field unchanged (clearing still
sends null for the auto-tracking behaviour).
Docs:
- docs/routes.md defaults table: window_hours 24->48, threshold 3->5,
clear (2x)->(3x), max_hop_span (unlimited)->8. Previous four
entries now match the shipped defaults (some were stale from #316).
- docs/seeding.md YAML example: window_hours 24->48, threshold 3->5,
commented clear_threshold example 10->15 with 3x note.
No migration: packet_count_threshold uses Python-side default= (no
server_default), so existing rows keep their stored values. Existing
routes with clear_threshold=NULL continue to track via the new 3x
multiplier at evaluation time.
Change the new-route defaults so the create modal and API/CLI paths
pre-fill a 48h evaluation window and a max-hop-span of 8 instead of
the previous 24h / unlimited (∞).
- schemas/routes.py: RouteCreate + RoutePreviewRequest defaults
- models/route.py: SQLAlchemy INSERT-time defaults
- collector/cli.py: YAML import fallbacks
- web/static/js/spa/pages/routes.js: new-route modal pre-fills 48/8
No migration: window_hours/max_hop_span use Python-side default=
(no server_default), so existing rows are untouched. Clearing the
max-hop-span field in the UI still sends null (no cap).
Routes list was sorted only by from_label; routes sharing the same
From appeared in backend-returned order, which looked random. Add
to_label as a secondary tiebreaker in the localeCompare comparator.
Case-sensitivity preserved (no behavior change for the primary key).
Adds a small DaisyUI loading-spinner to the primary action button and
disables both action buttons (Save/Delete + Cancel) while the request
is in flight. Prevents double-submit on slow operations like route
create/update, and prevents closing a modal mid-request which leaves
modalState in a confusing state.
Two patterns in the SPA, kept idiomatic to each:
- channels.js + routes.js (lit-html state-driven): thread a 'saving'
boolean from modalState into the modal renderers, which add
?disabled + a leading <span class="loading loading-spinner
loading-sm"></span> when set. Handlers set the flag and re-render
before await; clear and re-render in the catch.
- node-detail.js (native <dialog> + imperative listeners): add an
id to the Save button so the handler can toggle .disabled and swap
.innerHTML on the button element directly, restored in a finally.
Covers all six modal action buttons: channel add/edit + delete,
route add/edit + delete, node-tag edit + delete.
- docker-compose.yml: passthrough ROUTE_HISTORY_BACKFILL_INTERVAL_SECONDS
alongside the existing ROUTE_EVALUATOR_INTERVAL_SECONDS (operators
setting the var in .env had no effect without the passthrough).
- configuration.md: document the new collector var in the Collector table.
- upgrading.md (Route Health Monitoring): correct the migration summary
(seven tables, not five) and add a paragraph on the precompute
behaviour; add the new env var row to the table.
- upgrading.md (Dashboard / Route cache consolidation): drop the stale
'REDIS_CACHE_TTL_DASHBOARD raised to 3600' claim — the precompute
revert lowered it back to 300 s. Renamed the section to
'API endpoint reorganization' and retained the two ship-true changes
(ROUTE_DETAIL TTL removed, /dashboard/recent-activity split).
Replaces ec40c67c8c83, 6b3430fd84f4 and cf8dd7eaba9b (never deployed
to production) with one migration that builds the final route-health
schema directly on top of the production head 57bb65130b97.
The consolidated migration creates the seven route tables, adds the
nullable event_hash column to raw_packets, and runs three idempotent
backfills: packet_path_hops from raw_packets.decoded, nodes dedup by
public_key, and route_result_history + quality_avg for enabled routes.
On a freshly-restored production snapshot the route-history backfill
is a no-op (no routes exist yet) but is retained for idempotency
against dev backups that do have routes.
Commit 37f8d7a re-introduced emoji extraction in observerFilterBadges
after d1a3605 had refactored the iteration variable from `n` (node
object) to `area` (plain string). Three references to n._displayName
and n.public_key were left behind, throwing ReferenceError the moment
any observer had an area tag.
Adverts and Messages pages surface this as a warning badge with the
error message; the early return hides the bug on deployments with no
area-tagged observers.
Replace n.X with area (the iteration variable). Emoji extraction and
label fallback now operate on the area string directly.
Persist route health derivations so the API layer no longer recomputes
them on every request. Two new tables (route_result_history,
route_recent_matches) plus a quality_avg column on route_results back
the dashboard strip and per-route history endpoints.
Collector:
- run_evaluation (60s) writes snapshot + quality_avg + recent_matches
- run_history_backfill (hourly) recomputes completed-day buckets
- subscriber wires a dedicated backfill scheduler thread
API:
- dashboard routes-overview bulk-loads via single history SELECT
- routes detail/detail history read from precomputed tables with
live-compute fallback
- POST/PUT on routes upserts recent_matches and quality_avg inline
- dashboard cache TTL lowered from 1h to 5m (invalidation-aware)
Config: route_history_backfill_interval_seconds=3600,
redis_cache_ttl_dashboard default 3600 -> 300
The overall health badge on route cards now shows the rolling 7-day
average tier instead of the latest window-hours snapshot, so flapping
routes that are currently up still appear marginal/failing if the
week's mean warrants it. Same averaging drives the dashboard Route
Health widget's summary dot, the routes page summary strip counts,
and (already) the Route Trends chart line colors.
- compute_average_quality() in collector/routes.py (0/1/2 mean,
thresholds 1.5/0.75, empty-history fallback) — kept in sync with
the averageRouteTier JS helper in charts.js
- RouteRead / RouteDetail gain a 'quality_avg' field
- list/get/update handlers compute it per route; create skips (no
meaningful history yet) and the frontend falls back to
route_result.quality for brand-new routes
- diagnosis tooltip unchanged (still current-snapshot state text)
Adds Route Trends chart and Route Health strip grid to the dashboard,
backs them with a new GET /dashboard/routes-overview endpoint, and
consolidates the Redis cache layer (removes ROUTE_DETAIL TTL, raises
DASHBOARD default to 1h, splits recent-activity into its own short-TTL
endpoint so Recent panels stay fresh while aggregate counts cache 1h).
Reported symptom: user edits a Route (e.g. lowers packet_count_threshold
from 6 to 3), and the routes list card still shows the old threshold for
~30s. The cache stack was exonerated — x-cache: MISS on the stale GET,
response body had the new packet_count_threshold=3, but route_result.
threshold remained at 6.
Root cause was neither HTTP cache nor Redis cache. The list card's stats
row (renderStatsRow in routes.js:82) displays route_result.threshold /
effective_clear / matched_count, which are persisted by the background
route_evaluator on a 30-60s schedule — separate DB row from the route's
direct fields. The PUT handler updated the route row immediately but
never triggered a re-evaluation, so route_result carried the stale
snapshot from the prior evaluator cycle until the next sweep.
Fix: add _reevaluate_route(session, route) helper that runs evaluate_route
+ upsert_route_result synchronously after the route commit. Called from
create_route (initial evaluation) and update_route (refresh on every
config change). Disabled routes short-circuit (no point evaluating a
route that's turned off). One bounded DB scan per write — same cost as
a single-route evaluator tick.
After this fix, the PUT response itself carries a fresh route_result
reflecting the new packet_count_threshold / clear_threshold, and the
next GET /api/v1/routes returns it. The list card updates on the very
next render cycle.
Tests:
- test_update_threshold_immediately_reflects_in_route_result: seeds a
stale RouteResult with the OLD threshold, sends a PUT, asserts the
response's route_result.threshold/effective_clear now match the new
route config.
- test_disabled_route_does_not_trigger_evaluation: monkeypatches
evaluate_route to a spy, asserts it's never called for disabled routes.
After PR #311's invalidation wiring and HTTP policy fix, a user reported
the routes list still showed stale data for ~30s after a PUT. The HAR
they sent actually showed the fix working (x-cache: MISS after PUT,
fresh data returned) but had been captured with DevTools' 'Disable
cache' enabled — so it didn't represent normal browsing. We had four
competing hypotheses and no way to pick between them.
Add structured INFO logging at the two points that matter:
* meshcore_hub.api.cache_invalidation._drop now emits
'Cache invalidate start: prefix=... backend=...' and
'Cache invalidate ok: prefix=...'. The backend= field
distinguishes RedisCacheBackend from NullCache in one glance,
catching 'REDIS_ENABLED is actually false' cases.
* meshcore_hub.common.redis.RedisCacheBackend.delete now emits
'Redis cache delete: prefix=... full_prefix=... keys_deleted=N
scan_iterations=N'. keys_deleted=0 after a mutation that should
have invalidated entries is the smoking gun for a cache-key
mismatch between the store path (key_builder) and the delete
path (prefix glob).
* NullCache.delete emits a DEBUG line for the same reason.
The error paths are also enriched with full_prefix for greppability.
No behavioral changes — invalidation still swallows errors so cache
outages never break a write.
After deploy, a single route-edit repro will produce log output whose
shape (start/ok/warning/missing, keys_deleted count) unambiguously
identifies which of the four hypotheses is correct, so we can write
a targeted fix instead of guessing.
After PR #311 wired server-side invalidation, a user reported that editing
a Route still showed old values on the routes list for ~30s. Root cause:
the api_cache_middleware emitted 'Cache-Control: private, max-age=30' on
@cached GETs, which let the browser serve its local HTTP cache copy
without revalidating. The Redis invalidation fired correctly but never
mattered — the browser never asked the server.
Switch all /api/* GET responses to 'private, no-cache' (synonymous with
'max-age=0, must-revalidate'). The browser now sends If-None-Match on
every navigation; the server answers 304 when Redis is warm and unchanged
(cheap — no body) or 200 after an invalidation.
The per-endpoint Redis TTL (30s default, 300s dashboard/route-detail)
still bounds cache.set lifetime; only the HTTP-layer max-age disappears.
Most navigations are still 304s, so the cost is one tiny round-trip per
page load while guaranteeing freshness after any mutation.
Adds an end-to-end regression test (test_routes_list_refresh_after_mutation)
modelling the exact reported scenario: cache-fill, conditional 304,
PUT mutation, conditional 200 with fresh body.
Mutation handlers (POST/PUT/DELETE) on channels, routes, user profiles,
node tags, and adoptions now drop the corresponding Redis cache entries
after commit so the UI reflects changes on the next page load instead of
waiting for the 30s/300s TTL.
Adds meshcore_hub.api.cache_invalidation with seven helpers that
encapsulate the two cache-key formats (endpoint-name keys like 'nodes:'
vs URL-path keys like '/api/v1/channels:') and swallow backend errors
so a cache outage never breaks a successful write. The helper is a
no-op when Redis is disabled.
Cross-entity embeddings are covered: node-tag writes invalidate nodes,
messages, advertisements, and dashboard; adoption writes invalidate
nodes, profiles, advertisements, and dashboard.
CLI/collector mutations remain TTL-bound (infrequent, operator-driven).
Bundles four independent improvements to API caching and route evaluator
accuracy that all touch the same files in the cache/routes stack.
## HTTP Cache-Control headers (api_cache_control_enabled)
The API now emits HTTP Cache-Control on every /api/v1/* response and
handles ETag / If-None-Match -> 304 Not Modified on @cached endpoints.
- @cached GETs: private, max-age=<redis_ttl> + strong SHA-256 ETag.
Matching If-None-Match returns 304 with empty body.
- Uncached GETs under /api/v1/*: private, max-age=0, must-revalidate.
- POST/PUT/DELETE/PATCH and /health*: no-store.
- All private (several @cached endpoints are role-aware).
- @cached decorator stores new envelope {body, etag} in Redis; legacy
bare-JSON entries still read (hashed on the fly) and auto-migrate on
the next miss.
- X-Cache: HIT|MISS observability header is always emitted.
- Kill switch: API_CACHE_CONTROL_ENABLED (default true).
## Dashboard cache TTL bump (30s -> 300s)
REDIS_CACHE_TTL_DASHBOARD default raised from 30s to 300s. Covers all
/dashboard/* endpoints and /api/v1/routes/{id}/history. These return
trend/aggregation data where minute-level staleness is invisible but
every MISS costs seconds of SQL/aggregation. Also resolves reported
short-TTL behaviour on the per-route healthchart (which shares this
setting). Operators with explicit env var: unchanged. Old Redis entries
expire naturally within <=30s; no flush needed.
## Route recent-packets hop truncation
In the routes page expanded card, path hops are now truncated to
2 + ellipsis + 2 when length > 5, matching the packet-detail page
styling. Reuses the existing packets.hops_hidden i18n key as the
tooltip on the ellipsis badge.
## Route event-hash deduplication
A single advert/message/telemetry/trace retransmitted or flooded
through the mesh produces one RawPacket per on-air copy, each with a
fresh wire packet_hash. Counting those as distinct matches let a single
underlying event satisfy packet_count_threshold within seconds and
biased route health.
- Add nullable event_hash column to raw_packets and packet_path_hops
(alembic 6b3430fd84f4). No backfill; legacy rows keep NULL and the
evaluator falls back to wire packet_hash until they age out of
window_hours.
- Subscriber denormalizes the structured event's event_hash onto the
captured raw packet after handler dispatch.
- Route evaluator prefers event_hash when set, collapsing all
receptions of the same underlying event into one match.
The route card header wrapped onto two lines when from/to labels were
long. Replace the inline "from -> to" title with a two-row CSS grid
(auto | 1fr) so labels align vertically, each preceded by a dedicated
SVG icon (iconRouteFrom/iconRouteTo: arrow departing/arriving at a
dot). Long labels now ellipsize with a title tooltip.
Move the reversibility indicator (reversible <-> vs one-way ->) out of
the title into a ghost badge beside the quality badge, where it no
longer fights the grid layout.
Also nudge spacing between "Recent Packets" and its path rows (mt-1 ->
mt-2) for a little breathing room.
Squashes four formerly-separate tail migrations into a single migration
(ec40c67c8c83) that builds the final schema directly off 57bb65130b97:
- 8f2a3c4d5e6f route health tables + packet_path_hops backfill
- c0d1e2f3a4b5 deduplicate nodes by public_key (idempotent data fix)
- 76513f4c57e9 ix_packet_path_hops_received_at index
- 71f6e01e4bf6 rename degraded_threshold -> clear_threshold
The rename runs as a no-op: routes.clear_threshold and
route_results.effective_clear are born with their final names. The
received_at index is created with the table, and the node dedup is
preserved verbatim (no-op on a clean DB).
Also drops the redundant uq_route_results_route_id unique constraint:
route_results.route_id now has a single unique index matching the model,
eliminating a pre-existing autogenerate drift.
Net schema is identical to running all four migrations. Verified with a
fresh alembic upgrade head + alembic check (no drift) on SQLite.
Test data used a hardcoded _NOW (2026-07-12 12:00 UTC) while
run_evaluation used real datetime.now(), causing a time-window flake
once wall-clock time exceeded _NOW + window_hours (24h). Tests now
pass now=_NOW explicitly; production callers are unaffected (defaults
to datetime.now(timezone.utc) when not provided).
The Route UI's observer search was hitting GET /api/v1/nodes without
the observer filter, showing all nodes instead of only observer nodes.
The backend already supports ?observer=true via an indexed column —
just needed to pass the param from handleObsSearch and handleObsKeydown.
Louis King