Operators can now create, edit, and delete routes (previously admin-only).
A user may never scope a route above their own role tier: an operator
creating/editing an admin-visibility route is rejected (403 on the
visibility value, 404 on touching an existing higher-visibility route),
preventing them from creating routes they could then never see or modify.
- routes.py: RequireAdmin -> RequireOperatorOrAdmin on create/update/delete;
add visibility-cap enforcement helpers reusing the existing
resolve_user_role / VISIBILITY_LEVELS ladder
- web/app.py: proxy access map admits operator for routes POST/PUT/DELETE
- Routes.tsx: canManage gate (admin||operator) on Add/Edit/Delete; visibility
<select> filters options by caller tier so operators never see 'admin'
- tests: operator-tier coverage (create/update/delete at/below/above level),
proxy access-map assertion, vitest role-gating + filtered select
- e2e: mint operator session + routes-operator spec
- docs: routes.md + auth.md operator/visibility-cap notes
Migrate the React SPA off the bespoke useApiFetch hook and raw useEffect fetches to TanStack Query: useQuery/useQueries for reads, useMutation + invalidateQueries for writes; a central query-key factory and invalidation helpers mirroring the backend cache_invalidation prefixes (channels/routes/nodes/messages/profiles/dashboard/adoptions); polling via refetchInterval (useAutoRefresh now returns pause state only); RouteCard self-fetches its detail/history, dropping the hand-rolled caches. Adds QueryClientProvider in App, a renderWithProviders test helper, and deletes useApiFetch.
Also consolidates recurring UI into reusable components (Breadcrumbs, ListToolbar, Modal/ConfirmDialog, NotFoundState, TimeAgo, Definition, CopyableValue, MeshQrCode, Badges, SectionGroup, PageHeader) and fixes the FilterForm clear button, merged toggle wrappers, and role-aware channels/messages cache keys so client invalidation matches the server.
Verified: tsc --noEmit clean, vitest 113 passed, pytest test_web 251 + test_cache 119 passed, pre-commit --all-files green.
Persist route health derivations so the API layer no longer recomputes
them on every request. Two new tables (route_result_history,
route_recent_matches) plus a quality_avg column on route_results back
the dashboard strip and per-route history endpoints.
Collector:
- run_evaluation (60s) writes snapshot + quality_avg + recent_matches
- run_history_backfill (hourly) recomputes completed-day buckets
- subscriber wires a dedicated backfill scheduler thread
API:
- dashboard routes-overview bulk-loads via single history SELECT
- routes detail/detail history read from precomputed tables with
live-compute fallback
- POST/PUT on routes upserts recent_matches and quality_avg inline
- dashboard cache TTL lowered from 1h to 5m (invalidation-aware)
Config: route_history_backfill_interval_seconds=3600,
redis_cache_ttl_dashboard default 3600 -> 300
Adds Route Trends chart and Route Health strip grid to the dashboard,
backs them with a new GET /dashboard/routes-overview endpoint, and
consolidates the Redis cache layer (removes ROUTE_DETAIL TTL, raises
DASHBOARD default to 1h, splits recent-activity into its own short-TTL
endpoint so Recent panels stay fresh while aggregate counts cache 1h).
After PR #311's invalidation wiring and HTTP policy fix, a user reported
the routes list still showed stale data for ~30s after a PUT. The HAR
they sent actually showed the fix working (x-cache: MISS after PUT,
fresh data returned) but had been captured with DevTools' 'Disable
cache' enabled — so it didn't represent normal browsing. We had four
competing hypotheses and no way to pick between them.
Add structured INFO logging at the two points that matter:
* meshcore_hub.api.cache_invalidation._drop now emits
'Cache invalidate start: prefix=... backend=...' and
'Cache invalidate ok: prefix=...'. The backend= field
distinguishes RedisCacheBackend from NullCache in one glance,
catching 'REDIS_ENABLED is actually false' cases.
* meshcore_hub.common.redis.RedisCacheBackend.delete now emits
'Redis cache delete: prefix=... full_prefix=... keys_deleted=N
scan_iterations=N'. keys_deleted=0 after a mutation that should
have invalidated entries is the smoking gun for a cache-key
mismatch between the store path (key_builder) and the delete
path (prefix glob).
* NullCache.delete emits a DEBUG line for the same reason.
The error paths are also enriched with full_prefix for greppability.
No behavioral changes — invalidation still swallows errors so cache
outages never break a write.
After deploy, a single route-edit repro will produce log output whose
shape (start/ok/warning/missing, keys_deleted count) unambiguously
identifies which of the four hypotheses is correct, so we can write
a targeted fix instead of guessing.
After PR #311 wired server-side invalidation, a user reported that editing
a Route still showed old values on the routes list for ~30s. Root cause:
the api_cache_middleware emitted 'Cache-Control: private, max-age=30' on
@cached GETs, which let the browser serve its local HTTP cache copy
without revalidating. The Redis invalidation fired correctly but never
mattered — the browser never asked the server.
Switch all /api/* GET responses to 'private, no-cache' (synonymous with
'max-age=0, must-revalidate'). The browser now sends If-None-Match on
every navigation; the server answers 304 when Redis is warm and unchanged
(cheap — no body) or 200 after an invalidation.
The per-endpoint Redis TTL (30s default, 300s dashboard/route-detail)
still bounds cache.set lifetime; only the HTTP-layer max-age disappears.
Most navigations are still 304s, so the cost is one tiny round-trip per
page load while guaranteeing freshness after any mutation.
Adds an end-to-end regression test (test_routes_list_refresh_after_mutation)
modelling the exact reported scenario: cache-fill, conditional 304,
PUT mutation, conditional 200 with fresh body.
Mutation handlers (POST/PUT/DELETE) on channels, routes, user profiles,
node tags, and adoptions now drop the corresponding Redis cache entries
after commit so the UI reflects changes on the next page load instead of
waiting for the 30s/300s TTL.
Adds meshcore_hub.api.cache_invalidation with seven helpers that
encapsulate the two cache-key formats (endpoint-name keys like 'nodes:'
vs URL-path keys like '/api/v1/channels:') and swallow backend errors
so a cache outage never breaks a successful write. The helper is a
no-op when Redis is disabled.
Cross-entity embeddings are covered: node-tag writes invalidate nodes,
messages, advertisements, and dashboard; adoption writes invalidate
nodes, profiles, advertisements, and dashboard.
CLI/collector mutations remain TTL-bound (infrequent, operator-driven).
Bundles four independent improvements to API caching and route evaluator
accuracy that all touch the same files in the cache/routes stack.
## HTTP Cache-Control headers (api_cache_control_enabled)
The API now emits HTTP Cache-Control on every /api/v1/* response and
handles ETag / If-None-Match -> 304 Not Modified on @cached endpoints.
- @cached GETs: private, max-age=<redis_ttl> + strong SHA-256 ETag.
Matching If-None-Match returns 304 with empty body.
- Uncached GETs under /api/v1/*: private, max-age=0, must-revalidate.
- POST/PUT/DELETE/PATCH and /health*: no-store.
- All private (several @cached endpoints are role-aware).
- @cached decorator stores new envelope {body, etag} in Redis; legacy
bare-JSON entries still read (hashed on the fly) and auto-migrate on
the next miss.
- X-Cache: HIT|MISS observability header is always emitted.
- Kill switch: API_CACHE_CONTROL_ENABLED (default true).
## Dashboard cache TTL bump (30s -> 300s)
REDIS_CACHE_TTL_DASHBOARD default raised from 30s to 300s. Covers all
/dashboard/* endpoints and /api/v1/routes/{id}/history. These return
trend/aggregation data where minute-level staleness is invisible but
every MISS costs seconds of SQL/aggregation. Also resolves reported
short-TTL behaviour on the per-route healthchart (which shares this
setting). Operators with explicit env var: unchanged. Old Redis entries
expire naturally within <=30s; no flush needed.
## Route recent-packets hop truncation
In the routes page expanded card, path hops are now truncated to
2 + ellipsis + 2 when length > 5, matching the packet-detail page
styling. Reuses the existing packets.hops_hidden i18n key as the
tooltip on the ellipsis badge.
## Route event-hash deduplication
A single advert/message/telemetry/trace retransmitted or flooded
through the mesh produces one RawPacket per on-air copy, each with a
fresh wire packet_hash. Counting those as distinct matches let a single
underlying event satisfy packet_count_threshold within seconds and
biased route health.
- Add nullable event_hash column to raw_packets and packet_path_hops
(alembic 6b3430fd84f4). No backfill; legacy rows keep NULL and the
evaluator falls back to wire packet_hash until they age out of
window_hours.
- Subscriber denormalizes the structured event's event_hash onto the
captured raw packet after handler dispatch.
- Route evaluator prefers event_hash when set, collapsing all
receptions of the same underlying event into one match.
The API response cache key was built from request.query_params.items(),
which in Starlette keeps only the last value of a repeated key. Repeated
params like observed_by (?observed_by=A&observed_by=B) therefore collapsed
to observed_by=B in the key, colliding with any other filter set sharing the
same last value. The first response cached under that key was then served
for the whole TTL, making observer-filtered Messages/Adverts return stale,
wrong results (e.g. an A-only message vanishing when B was also enabled).
Use multi_items() so all repeated values are included, sorted by the full
(key, value) tuple so order-independent filter sets map to one key. Add
regression tests covering preservation, order-independence, and the
{A,B} vs {B} collision case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Louis King