diff --git a/.env.example b/.env.example index 0148eed..4a950cc 100644 --- a/.env.example +++ b/.env.example @@ -411,11 +411,12 @@ PROMETHEUS_PORT=9090 # Default cache TTL in seconds (matches web auto-refresh interval) # REDIS_CACHE_TTL=30 -# Cache TTL for dashboard endpoints (seconds; covers /dashboard/* and /routes/{id}/history) -# REDIS_CACHE_TTL_DASHBOARD=300 - -# Cache TTL for /routes/{id} detail endpoint (seconds) -# REDIS_CACHE_TTL_ROUTE_DETAIL=300 +# Cache TTL for dashboard endpoints, /routes/{id} detail, and per-route +# health history (seconds). Trend/aggregation data tolerates much longer +# staleness than the default TTL. The Recent Adverts / Recent Channel +# Messages widgets live on a separate /dashboard/recent-activity endpoint +# that always uses REDIS_CACHE_TTL (above) so they stay fresh. +# REDIS_CACHE_TTL_DASHBOARD=3600 # Emit HTTP Cache-Control on /api/v1/* responses + ETag/If-None-Match on # cached endpoints. The policy is `private, no-cache` on GETs (forces diff --git a/docker-compose.yml b/docker-compose.yml index 531fb57..b756362 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -320,8 +320,7 @@ services: - REDIS_PASSWORD=${REDIS_PASSWORD:-} - REDIS_KEY_PREFIX=${REDIS_KEY_PREFIX:-hub} - REDIS_CACHE_TTL=${REDIS_CACHE_TTL:-30} - - REDIS_CACHE_TTL_DASHBOARD=${REDIS_CACHE_TTL_DASHBOARD:-300} - - REDIS_CACHE_TTL_ROUTE_DETAIL=${REDIS_CACHE_TTL_ROUTE_DETAIL:-300} + - REDIS_CACHE_TTL_DASHBOARD=${REDIS_CACHE_TTL_DASHBOARD:-3600} - API_CACHE_CONTROL_ENABLED=${API_CACHE_CONTROL_ENABLED:-true} # Spam detection: hide-filter switch + threshold (bridged from # FEATURE_SPAM_DETECTION so it tracks the collector and web toggle). On by diff --git a/docs/configuration.md b/docs/configuration.md index fec99c6..5304a3e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -61,9 +61,8 @@ Optional Redis-backed caching for API responses. When disabled or unavailable, t | `REDIS_DB` | `0` | Redis database number | | `REDIS_PASSWORD` | _(none)_ | Redis password (optional) | | `REDIS_KEY_PREFIX` | `hub` | Cache key prefix for multi-instance isolation | -| `REDIS_CACHE_TTL` | `30` | Default cache TTL in seconds | -| `REDIS_CACHE_TTL_DASHBOARD` | `300` | Cache TTL for `/dashboard/*` endpoints and `/routes/{id}/history` (seconds). Trend/aggregation data tolerates longer staleness than the default TTL. | -| `REDIS_CACHE_TTL_ROUTE_DETAIL` | `300` | Cache TTL for `/routes/{id}` detail endpoint in seconds | +| `REDIS_CACHE_TTL` | `30` | Default cache TTL in seconds. Also covers `GET /dashboard/recent-activity` (Recent Adverts / Recent Channel Messages widgets) so those panels stay fresh independently of the longer dashboard TTL below. | +| `REDIS_CACHE_TTL_DASHBOARD` | `3600` | Cache TTL in seconds for `/dashboard/*` endpoints (except `/recent-activity`), `/routes/{id}` detail, and `/routes/{id}/history`. Trend/aggregation data tolerates much longer staleness than the default TTL. | ### HTTP Cache-Control diff --git a/docs/upgrading.md b/docs/upgrading.md index 0355f06..5201785 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -59,19 +59,34 @@ To suppress all client-side caching directives (e.g. for debugging): The `X-Cache: HIT|MISS` observability header is unaffected — it is still emitted whenever a request flows through the `@cached` decorator, regardless of this setting. -### Dashboard Cache TTL default raised (30s → 300s) +### Dashboard / Route cache consolidation (1h dashboard TTL, dedicated route-detail TTL removed) -The default for `REDIS_CACHE_TTL_DASHBOARD` has been raised from **30 s** to **300 s** (5 minutes). This setting covers all `/dashboard/*` endpoints (`stats`, `activity`, `packet-activity`, `packet-breakdown`, `message-activity`, `node-count`) **and** the per-route health strip `GET /api/v1/routes/{id}/history`. These all return trend/aggregation data where minute-level staleness is invisible to operators but every cache miss costs seconds of SQL/aggregation work — the old 30 s default was leaving most of that work unpaid on typical dashboards. +Two related changes to the Redis cache layer: -| Variable | Old default | New default | -| --- | --- | --- | -| `REDIS_CACHE_TTL_DASHBOARD` | `30` | `300` | +1. **`REDIS_CACHE_TTL_DASHBOARD` default raised from 300 s to 3600 s (1 hour).** This setting now covers all `/dashboard/*` endpoints **and** `/routes/{id}` detail **and** `/routes/{id}/history` — trend/aggregation data where minute-level staleness is invisible to operators but every cache miss costs seconds of SQL/aggregation work. The 5-minute window shipped earlier in v0.16.0 was still leaving most of that work unpaid on busy meshes. -- **Operators who already set the env var explicitly:** no change — your value still wins. -- **Operators on the default:** silently bumped to 5 min staleness on the dashboard and route health strips. If you relied on the old 30 s behaviour (e.g. for development or tight freshness SLOs), set `REDIS_CACHE_TTL_DASHBOARD=30` to restore it. -- **Existing Redis entries at deploy time:** keep their original TTL and expire naturally within ≤ 30 s. No flush is required. -- **Browser `max-age`:** the HTTP `Cache-Control: max-age=` emitted by the middleware follows the Redis TTL, so browser-side caching on these endpoints also extends to 5 min. Clicking around the dashboard will be instant for that window. A hard refresh (`Ctrl+Shift+R`) bypasses this if you need the latest data. -- **Side-effect fix:** the reported short-TTL behaviour on the per-route healthchart (which shares this setting) is resolved. +2. **`REDIS_CACHE_TTL_ROUTE_DETAIL` removed.** The dedicated 300 s TTL for `/routes/{id}` detail has been folded into `REDIS_CACHE_TTL_DASHBOARD` (so the route detail page now also serves from the 1 h cache). Operators who had set `REDIS_CACHE_TTL_ROUTE_DETAIL` explicitly should delete that line from their `.env` — the variable is now ignored. + +3. **`GET /dashboard/recent-activity` split out of `GET /dashboard/stats`.** The Recent Adverts and Recent Channel Messages widgets used to live as fields inside `/dashboard/stats`, so they inherited the long dashboard TTL and went stale for the full window. They now have their own endpoint, `/dashboard/recent-activity`, which is cached at the **default** `REDIS_CACHE_TTL` (30 s) — so the Recent panels stay fresh even while the aggregate counts alongside them are cached for an hour. The two fields were **removed from the `/dashboard/stats` response**: + + | Field | Old location | New location | + | --------------------------- | ------------------------- | ------------------------------------- | + | `recent_advertisements` | `/dashboard/stats` field | `/dashboard/recent-activity` field | + | `channel_messages` | `/dashboard/stats` field | `/dashboard/recent-activity` field | + + `/dashboard/stats` still returns `channel_message_counts` (the per-channel counts); only the message bodies moved. The SPA's dashboard page now fires one extra parallel request to `/dashboard/recent-activity`. `home.js` only used the counts and needs no change — it silently benefits from the longer `/dashboard/stats` TTL. + +| Variable | Old default | New default | +| ----------------------------- | ----------- | ----------- | +| `REDIS_CACHE_TTL_DASHBOARD` | `300` | `3600` | +| `REDIS_CACHE_TTL_ROUTE_DETAIL`| `300` | _(removed)_ | + +- **Operators who already set `REDIS_CACHE_TTL_DASHBOARD` explicitly:** no change — your value still wins. +- **Operators on the default:** silently bumped from 5 min to 1 h staleness on dashboard aggregates, route detail, and route health strips. If you relied on the previous 5 min freshness (e.g. for development), set `REDIS_CACHE_TTL_DASHBOARD=300` (or lower) to restore it. +- **Operators who set `REDIS_CACHE_TTL_ROUTE_DETAIL`:** delete the line — the variable is now ignored, the route detail page uses `REDIS_CACHE_TTL_DASHBOARD` like everything else in this group. +- **Existing Redis entries at deploy time:** keep their original TTL and expire naturally. No flush is required. +- **Browser `max-age`:** the HTTP `Cache-Control: max-age=` emitted by the middleware follows the Redis TTL, so browser-side caching on these endpoints also extends to 1 h. The Recent Adverts / Recent Channel Messages widgets are unaffected because `/dashboard/recent-activity` still uses the short default TTL. +- **External API consumers** reading `recent_advertisements` / `channel_messages` from `/dashboard/stats`: switch to `/dashboard/recent-activity` (same field shapes, same role-visibility filtering). ## v0.15.0 diff --git a/src/meshcore_hub/api/app.py b/src/meshcore_hub/api/app.py index 4b358ff..a2569bc 100644 --- a/src/meshcore_hub/api/app.py +++ b/src/meshcore_hub/api/app.py @@ -90,8 +90,7 @@ def create_app( redis_password: str | None = None, redis_key_prefix: str = "hub", redis_cache_ttl: int = 30, - redis_cache_ttl_dashboard: int = 300, - redis_cache_ttl_route_detail: int = 300, + redis_cache_ttl_dashboard: int = 3600, api_cache_control_enabled: bool = True, spam_detection_enabled: bool = False, spam_score_threshold: float = 0.65, @@ -120,8 +119,8 @@ def create_app( redis_password: Redis password (optional) redis_key_prefix: Prefix for all cache keys redis_cache_ttl: Default cache TTL in seconds - redis_cache_ttl_dashboard: Cache TTL for dashboard endpoints - redis_cache_ttl_route_detail: Cache TTL for /routes/{id} detail endpoint + redis_cache_ttl_dashboard: Cache TTL in seconds for dashboard endpoints, + /routes/{id} detail, and per-route health history api_cache_control_enabled: Emit HTTP Cache-Control on /api/v1/* and ETag/If-None-Match handling on @cached endpoints. @@ -159,7 +158,6 @@ def create_app( app.state.redis_key_prefix = redis_key_prefix app.state.redis_cache_ttl = redis_cache_ttl app.state.redis_cache_ttl_dashboard = redis_cache_ttl_dashboard - app.state.redis_cache_ttl_route_detail = redis_cache_ttl_route_detail app.state.api_cache_control_enabled = api_cache_control_enabled app.state.spam_detection_enabled = spam_detection_enabled app.state.spam_score_threshold = spam_score_threshold @@ -338,7 +336,6 @@ def create_app_from_env() -> FastAPI: redis_key_prefix=settings.redis_key_prefix, redis_cache_ttl=settings.redis_cache_ttl, redis_cache_ttl_dashboard=settings.redis_cache_ttl_dashboard, - redis_cache_ttl_route_detail=settings.redis_cache_ttl_route_detail, api_cache_control_enabled=settings.api_cache_control_enabled, spam_detection_enabled=settings.spam_detection_enabled, spam_score_threshold=settings.spam_score_threshold, diff --git a/src/meshcore_hub/api/cache_invalidation.py b/src/meshcore_hub/api/cache_invalidation.py index d198587..62d0ac7 100644 --- a/src/meshcore_hub/api/cache_invalidation.py +++ b/src/meshcore_hub/api/cache_invalidation.py @@ -89,8 +89,14 @@ def invalidate_routes(request: Request) -> None: All three endpoints share the ``/api/v1/routes`` URL-path prefix in their cache keys (the ``{id}`` and ``{id}/history`` sub-paths glob-match the same SCAN), so a single ``delete`` covers them. + + The dashboard's ``GET /dashboard/routes-overview`` endpoint also embeds + per-route state and history, so it must be invalidated alongside the + per-route caches. That key lives under the ``dashboard/routes-overview`` + endpoint-name namespace (no ``key_builder``), so it needs its own drop. """ _drop(request, "/api/v1/routes") + _drop(request, "dashboard/routes-overview") def invalidate_nodes(request: Request) -> None: diff --git a/src/meshcore_hub/api/cli.py b/src/meshcore_hub/api/cli.py index 256ea36..d8dd0fc 100644 --- a/src/meshcore_hub/api/cli.py +++ b/src/meshcore_hub/api/cli.py @@ -173,16 +173,12 @@ import click @click.option( "--redis-cache-ttl-dashboard", type=int, - default=300, + default=3600, envvar="REDIS_CACHE_TTL_DASHBOARD", - help="Cache TTL for dashboard endpoints (seconds)", -) -@click.option( - "--redis-cache-ttl-route-detail", - type=int, - default=300, - envvar="REDIS_CACHE_TTL_ROUTE_DETAIL", - help="Cache TTL for /routes/{id} detail endpoint (seconds)", + help=( + "Cache TTL in seconds for dashboard endpoints, /routes/{id} detail, " + "and per-route health history" + ), ) @click.option( "--api-cache-control-enabled/--no-api-cache-control", @@ -239,7 +235,6 @@ def api( redis_key_prefix: str, redis_cache_ttl: int, redis_cache_ttl_dashboard: int, - redis_cache_ttl_route_detail: int, api_cache_control_enabled: bool, reload: bool, workers: int, @@ -299,8 +294,7 @@ def api( click.echo(f"Redis key prefix: {redis_key_prefix}") click.echo( f"Redis cache TTL: {redis_cache_ttl}s " - f"(dashboard: {redis_cache_ttl_dashboard}s, " - f"route detail: {redis_cache_ttl_route_detail}s)" + f"(dashboard: {redis_cache_ttl_dashboard}s)" ) click.echo(f"API Cache-Control enabled: {api_cache_control_enabled}") click.echo(f"Reload mode: {reload}") @@ -364,7 +358,6 @@ def api( redis_key_prefix=redis_key_prefix, redis_cache_ttl=redis_cache_ttl, redis_cache_ttl_dashboard=redis_cache_ttl_dashboard, - redis_cache_ttl_route_detail=redis_cache_ttl_route_detail, api_cache_control_enabled=api_cache_control_enabled, ) diff --git a/src/meshcore_hub/api/routes/dashboard.py b/src/meshcore_hub/api/routes/dashboard.py index 680e10c..0dcf832 100644 --- a/src/meshcore_hub/api/routes/dashboard.py +++ b/src/meshcore_hub/api/routes/dashboard.py @@ -9,6 +9,7 @@ from sqlalchemy.sql.elements import ColumnElement from meshcore_hub.api.auth import RequireRead from meshcore_hub.api.cache import cached, sorted_query_string from meshcore_hub.api.channel_visibility import ( + VISIBILITY_LEVELS, get_max_visibility_level, get_visible_channel_indices, resolve_user_role, @@ -18,12 +19,15 @@ from meshcore_hub.api.observer_utils import ( fetch_observers_for_events, resolve_sender_names, ) +from meshcore_hub.collector.routes import evaluate_route_history +from meshcore_hub.common.config import get_collector_settings from meshcore_hub.common.models import ( Advertisement, Message, Node, NodeTag, RawPacket, + Route, UserProfile, ) from meshcore_hub.common.schemas.messages import ( @@ -35,8 +39,14 @@ from meshcore_hub.common.schemas.messages import ( MessageActivity, NodeCountHistory, PacketBreakdown, + RecentActivity, RecentAdvertisement, ) +from meshcore_hub.common.schemas.routes import ( + RouteDayQuality, + RouteOverviewEntry, + RoutesOverview, +) router = APIRouter() @@ -53,6 +63,29 @@ def _dashboard_msg_activity_key_builder(request: Request) -> str: return f"dashboard/message-activity:role={role}:{sorted_query_string(request)}" +def _dashboard_recent_activity_key_builder(request: Request) -> str: + """Role-scoped key for ``GET /dashboard/recent-activity``. + + The response filters channel messages by visibility, so the cache key + must vary by role — otherwise an anonymous GET would fill the cache + with a redacted response served to a subsequent admin GET. + """ + role = resolve_user_role(request) or "anonymous" + return f"dashboard/recent-activity:role={role}:{sorted_query_string(request)}" + + +def _dashboard_routes_overview_key_builder(request: Request) -> str: + """Role-scoped key for ``GET /dashboard/routes-overview``. + + The endpoint filters admin/operator-only routes by visibility, so the + cache key must vary by role — otherwise an anonymous GET could fill the + cache with a response that hides admin routes, and a subsequent admin + GET would receive that same redacted response. + """ + role = resolve_user_role(request) or "anonymous" + return f"dashboard/routes-overview:role={role}:{sorted_query_string(request)}" + + def _flood_only_filter( ad_model: type[Advertisement], ) -> ColumnElement[bool]: @@ -166,6 +199,89 @@ def get_stats( total_packets = packet_row[0] or 0 packets_7d = packet_row[1] or 0 + # Channel message counts (only visible channels). The recent messages + # themselves live on /dashboard/recent-activity so they can carry a + # shorter cache TTL than these aggregate counts. + channel_counts_query = ( + select(Message.channel_idx, func.count()) + .where(Message.message_type == "channel") + .where(Message.channel_idx.isnot(None)) + .where(Message.channel_idx.in_(visible_indices)) + .group_by(Message.channel_idx) + ) + channel_results = session.execute(channel_counts_query).all() + channel_message_counts = { + int(channel): int(count) for channel, count in channel_results + } + + from meshcore_hub.common.config import get_web_settings + + web_settings = get_web_settings() + operator_role = web_settings.oidc_role_operator + member_role = web_settings.oidc_role_member + test_role = web_settings.oidc_role_test + + # Operator + member counts in one query. The optional test-role exclusion + # is folded into each conditional branch. + operator_cond = UserProfile.roles.contains(operator_role) + member_cond = UserProfile.roles.contains(member_role) + if test_role: + not_test = ~UserProfile.roles.contains(test_role) + operator_cond = and_(operator_cond, not_test) + member_cond = and_(member_cond, not_test) + + profile_row = session.execute( + select( + func.sum(case((operator_cond, 1), else_=0)), + func.sum(case((member_cond, 1), else_=0)), + ).select_from(UserProfile) + ).one() + total_operators = profile_row[0] or 0 + total_members = profile_row[1] or 0 + + return DashboardStats( + total_nodes=total_nodes, + active_nodes=active_nodes, + total_messages=total_messages, + messages_today=messages_today, + messages_7d=messages_7d, + total_advertisements=total_advertisements, + advertisements_24h=advertisements_24h, + advertisements_7d=advertisements_7d, + channel_message_counts=channel_message_counts, + total_operators=total_operators, + total_members=total_members, + total_packets=total_packets, + packets_7d=packets_7d, + ) + + +@router.get("/recent-activity", response_model=RecentActivity) +@cached( + "dashboard/recent-activity", + key_builder=_dashboard_recent_activity_key_builder, +) +def get_recent_activity( + _: RequireRead, + session: DbSession, + request: Request, +) -> RecentActivity: + """Get the recent-adverts and recent-channel-messages lists. + + Split out of ``GET /dashboard/stats`` so they can be cached at the + default ``redis_cache_ttl`` (30 s) instead of the much longer + ``redis_cache_ttl_dashboard`` that covers the aggregate counts — the + Recent Adverts / Recent Channel Messages panels are the only + dashboard widgets that need minute-level freshness. + + Role-visibility filtered the same way as the rest of the dashboard: + anonymous viewers see only public-channel messages, while operators + see every channel. Cache key is role-scoped accordingly. + """ + role = resolve_user_role(request) + max_level = get_max_visibility_level(role) + visible_indices = get_visible_channel_indices(session, max_level) + # Recent advertisements (last 10, flood-only) recent_ads = ( session.execute( @@ -241,22 +357,11 @@ def get_stats( for ad in recent_ads ] - # Channel message counts (only visible channels) - channel_counts_query = ( - select(Message.channel_idx, func.count()) - .where(Message.message_type == "channel") - .where(Message.channel_idx.isnot(None)) - .where(Message.channel_idx.in_(visible_indices)) - .group_by(Message.channel_idx) - ) - channel_results = session.execute(channel_counts_query).all() - channel_message_counts = { - int(channel): int(count) for channel, count in channel_results - } - - # Get latest 5 messages for each channel that has messages + # Channel messages for each visible channel (up to 5 latest each). + # The per-channel counts stay on /dashboard/stats (they're aggregate + # counts, not a "recent" list); only the message bodies live here. channel_messages: dict[int, list[ChannelMessage]] = {} - for channel_idx, _ in channel_results: + for channel_idx in visible_indices: messages_query = ( select(Message) .where(Message.message_type == "channel") @@ -265,6 +370,8 @@ def get_stats( .limit(5) ) channel_msgs = session.execute(messages_query).scalars().all() + if not channel_msgs: + continue # Look up sender names for these messages msg_prefixes = [m.pubkey_prefix for m in channel_msgs if m.pubkey_prefix] @@ -285,47 +392,9 @@ def get_stats( for m in channel_msgs ] - from meshcore_hub.common.config import get_web_settings - - web_settings = get_web_settings() - operator_role = web_settings.oidc_role_operator - member_role = web_settings.oidc_role_member - test_role = web_settings.oidc_role_test - - # Operator + member counts in one query. The optional test-role exclusion - # is folded into each conditional branch. - operator_cond = UserProfile.roles.contains(operator_role) - member_cond = UserProfile.roles.contains(member_role) - if test_role: - not_test = ~UserProfile.roles.contains(test_role) - operator_cond = and_(operator_cond, not_test) - member_cond = and_(member_cond, not_test) - - profile_row = session.execute( - select( - func.sum(case((operator_cond, 1), else_=0)), - func.sum(case((member_cond, 1), else_=0)), - ).select_from(UserProfile) - ).one() - total_operators = profile_row[0] or 0 - total_members = profile_row[1] or 0 - - return DashboardStats( - total_nodes=total_nodes, - active_nodes=active_nodes, - total_messages=total_messages, - messages_today=messages_today, - messages_7d=messages_7d, - total_advertisements=total_advertisements, - advertisements_24h=advertisements_24h, - advertisements_7d=advertisements_7d, + return RecentActivity( recent_advertisements=recent_advertisements, - channel_message_counts=channel_message_counts, channel_messages=channel_messages, - total_operators=total_operators, - total_members=total_members, - total_packets=total_packets, - packets_7d=packets_7d, ) @@ -633,3 +702,104 @@ def get_node_count_history( data.append(DailyActivityPoint(date=date_str, count=running)) return NodeCountHistory(days=days, data=data) + + +@router.get("/routes-overview", response_model=RoutesOverview) +@cached( + "dashboard/routes-overview", + ttl_setting="redis_cache_ttl_dashboard", + key_builder=_dashboard_routes_overview_key_builder, +) +def get_routes_overview( + _: RequireRead, + session: DbSession, + request: Request, + days: int = 7, +) -> RoutesOverview: + """Get an aggregate snapshot of route fleet health for the dashboard. + + Returns: + + - ``by_state``: route counts bucketed by their current ``state`` + (clear / marginal / failing / no_coverage / disabled). + - ``routes``: one entry per visible route carrying its current + ``state``/``quality``/``matched_count`` (from the latest + ``RouteResult`` written by the background evaluator) plus a + ``days``-long history (includes today) for the trend chart and + per-route strip grid. + + Role-visibility filtered the same way as ``GET /routes`` so members + don't see admin-only routes. ``days`` is clamped to the configured + raw-packet retention window so history queries can't scan purged data. + """ + days = min(days, 90) + retention = get_collector_settings().effective_raw_packet_retention_days + days = min(days, retention) + + role = resolve_user_role(request) + max_level = get_max_visibility_level(role) + + routes = session.execute(select(Route).order_by(Route.from_label)).scalars().all() + visible = [r for r in routes if VISIBILITY_LEVELS.get(r.visibility, 0) <= max_level] + + # State buckets — ``disabled`` for switched-off routes, otherwise the + # evaluator's last persisted state (falling back to ``no_coverage`` + # when no result exists yet, e.g. a freshly created route). + state_counts: dict[str, int] = {} + entries: list[RouteOverviewEntry] = [] + for route in visible: + history_tuples = evaluate_route_history( + session, route, days, include_today=True + ) + history = [ + RouteDayQuality(date=d, quality=q, state=s, matched_count=c) + for d, q, s, c in history_tuples + ] + + if not route.enabled: + state = "disabled" + quality = "disabled" + matched: int | None = None + elif route.route_result is not None: + state = route.route_result.state or "no_coverage" + quality = route.route_result.quality or "no_coverage" + matched = route.route_result.matched_count + else: + state = "no_coverage" + quality = "no_coverage" + matched = None + + state_counts[state] = state_counts.get(state, 0) + 1 + entries.append( + RouteOverviewEntry( + id=str(route.id), + from_label=route.from_label, + to_label=route.to_label, + visibility=route.visibility, + enabled=route.enabled, + state=state, + quality=quality, + matched_count=matched, + history=history, + ) + ) + + # Stable, UI-friendly bucket order. State values come from the + # evaluator's ``RouteState`` enum (``healthy`` / ``unhealthy`` / + # ``no_coverage``) plus the synthetic ``disabled`` we emit for + # switched-off routes. Any unknown state sorts after the known set. + preferred_order = [ + "healthy", + "unhealthy", + "no_coverage", + "disabled", + ] + by_state: list[BreakdownBucket] = [] + for label in preferred_order: + count = state_counts.pop(label, 0) + if count: + by_state.append(BreakdownBucket(label=label, count=count)) + for label, count in sorted(state_counts.items()): + by_state.append(BreakdownBucket(label=label, count=count)) + + return RoutesOverview(days=days, by_state=by_state, routes=entries) diff --git a/src/meshcore_hub/api/routes/routes.py b/src/meshcore_hub/api/routes/routes.py index 0df70c0..cad1d0f 100644 --- a/src/meshcore_hub/api/routes/routes.py +++ b/src/meshcore_hub/api/routes/routes.py @@ -244,7 +244,7 @@ def create_route( @router.get("/{route_id}", response_model=RouteDetail) @cached( "routes/{id}", - ttl_setting="redis_cache_ttl_route_detail", + ttl_setting="redis_cache_ttl_dashboard", key_builder=_routes_key_builder, ) def get_route( diff --git a/src/meshcore_hub/collector/routes.py b/src/meshcore_hub/collector/routes.py index 6761c8c..af70ab6 100644 --- a/src/meshcore_hub/collector/routes.py +++ b/src/meshcore_hub/collector/routes.py @@ -585,13 +585,20 @@ def evaluate_route_history( """Evaluate a route over *days* calendar-day buckets (oldest first). Returns a list of ``(date, quality, state, matched_count)`` tuples. - When *include_today* is True, an extra partial-day entry for today is - appended. For a disabled route, every day returns ``unknown`` / - ``no_coverage`` / ``0`` without hitting the DB. + When *include_today* is True, an extra entry is appended whose + ``quality``/``state``/``matched_count`` are computed via + :func:`evaluate_route` over the route's rolling ``window_hours`` window + (the same call used to populate ``route_result`` for the badge), so the + rightmost chart segment matches the card's badge. The ``date`` of that + entry is today's date — its underlying window may extend into yesterday + when ``window_hours`` > time elapsed since UTC midnight. Historical + segments remain UTC calendar-day buckets. For a disabled route, every + day returns ``unknown`` / ``no_coverage`` / ``0`` without hitting the DB. Performance: fetches only hops matching the route's expected prefixes (one DB query) plus one GROUP BY existence check, then partitions by - day in Python. + day in Python. When *include_today* is True, an additional bounded + ``evaluate_route`` call is made for the rolling-window segment. """ current = now or datetime.now(timezone.utc) today_midnight = current.replace(hour=0, minute=0, second=0, microsecond=0) @@ -667,7 +674,16 @@ def evaluate_route_history( last_prefix = expected[-1] last_prefix_end = _hex_prefix_end(last_prefix) - day_paths: list[dict[str, list[dict[str, Any]]]] = [{} for _ in range(total_days)] + # When include_today is set, the rightmost segment is evaluated via + # ``evaluate_route`` over the route's rolling ``window_hours`` window + # so that it matches ``route_result.quality`` (the badge). Historical + # segments remain UTC calendar-day buckets. We therefore partition + # packets only into the historical indices and append today separately. + historical_days = total_days - (1 if include_today else 0) + + day_paths: list[dict[str, list[dict[str, Any]]]] = [ + {} for _ in range(historical_days) + ] for rp_id, hops in all_paths.items(): for hop in hops: h = hop["node_hash"] @@ -679,18 +695,18 @@ def evaluate_route_history( or (last_prefix is not None and last_prefix <= h < last_prefix_end) ): continue - for i in range(total_days): + for i in range(historical_days): if day_starts[i] <= ra < day_ends[i]: if rp_id not in day_paths[i]: day_paths[i][rp_id] = hops break - # --- Evaluate each day (CPU-only, no DB) --- + # --- Evaluate each historical day (CPU-only, no DB) --- eff_clear = effective_clear_threshold(route) threshold = route.packet_count_threshold results: list[tuple[date, str, str, int]] = [] - for i in range(total_days): + for i in range(historical_days): matched_packets: set[str] = set() for hops in day_paths[i].values(): if _match_hops(hops, expected, route.max_hop_span, reversible): @@ -711,6 +727,14 @@ def evaluate_route_history( quality = derive_quality(state, matched_count, threshold, eff_clear) results.append((day_dates[i], quality, state, matched_count)) + # --- Today segment: rolling window, same call as the badge evaluator --- + if include_today: + rolling_start = current - timedelta(hours=route.window_hours) + today_state, today_quality, today_matched = evaluate_route( + session, route, rolling_start + ) + results.append((day_dates[-1], today_quality, today_state, today_matched)) + return results diff --git a/src/meshcore_hub/common/config.py b/src/meshcore_hub/common/config.py index 0f2e93b..ef7e6d3 100644 --- a/src/meshcore_hub/common/config.py +++ b/src/meshcore_hub/common/config.py @@ -418,16 +418,15 @@ class APISettings(CommonSettings): description="Default cache TTL in seconds", ) redis_cache_ttl_dashboard: int = Field( - default=300, + default=3600, description=( - "Cache TTL in seconds for dashboard endpoints and per-route " - "health history (trend/aggregation data tolerates longer staleness)" + "Cache TTL in seconds for dashboard endpoints, /routes/{id} detail " + "and per-route health history (trend/aggregation data tolerates " + "longer staleness than the default TTL). The Recent Adverts / " + "Recent Channel Messages widgets live on a separate " + "/dashboard/recent-activity endpoint cached at redis_cache_ttl." ), ) - redis_cache_ttl_route_detail: int = Field( - default=300, - description="Cache TTL for /routes/{id} detail endpoint (seconds)", - ) # HTTP Cache-Control headers api_cache_control_enabled: bool = Field( diff --git a/src/meshcore_hub/common/schemas/messages.py b/src/meshcore_hub/common/schemas/messages.py index 4842150..fadeab4 100644 --- a/src/meshcore_hub/common/schemas/messages.py +++ b/src/meshcore_hub/common/schemas/messages.py @@ -267,7 +267,12 @@ class ChannelMessage(BaseModel): class DashboardStats(BaseModel): - """Schema for dashboard statistics.""" + """Schema for dashboard aggregate statistics (counts only). + + The Recent Adverts / Recent Channel Messages lists used to live here + but were split into ``GET /dashboard/recent-activity`` so they can + carry a shorter cache TTL than the expensive aggregate counts. + """ total_nodes: int = Field(..., description="Total number of nodes") active_nodes: int = Field(..., description="Nodes active in last 24h") @@ -281,17 +286,10 @@ class DashboardStats(BaseModel): advertisements_7d: int = Field( default=0, description="Advertisements received in last 7 days" ) - recent_advertisements: list[RecentAdvertisement] = Field( - default_factory=list, description="Last 10 advertisements" - ) channel_message_counts: dict[int, int] = Field( default_factory=dict, description="Message count per channel", ) - channel_messages: dict[int, list[ChannelMessage]] = Field( - default_factory=dict, - description="Recent messages per channel (up to 5 each)", - ) total_operators: int = Field(default=0, description="Number of operator-role users") total_members: int = Field( default=0, description="Number of member-role users (excluding operators)" @@ -300,6 +298,25 @@ class DashboardStats(BaseModel): packets_7d: int = Field(default=0, description="Packets captured in last 7 days") +class RecentActivity(BaseModel): + """Schema for ``GET /dashboard/recent-activity``. + + Hosts the recent-adverts / recent-channel-messages lists that used to + be embedded in ``DashboardStats``. Cached at the default ``redis_cache_ttl`` + (30 s) so the dashboard's Recent panels stay fresh even while the + aggregate counts are cached at the longer ``redis_cache_ttl_dashboard``. + Channel visibility is role-scoped via the cache key builder. + """ + + recent_advertisements: list[RecentAdvertisement] = Field( + default_factory=list, description="Last 10 advertisements" + ) + channel_messages: dict[int, list[ChannelMessage]] = Field( + default_factory=dict, + description="Recent messages per channel (up to 5 each, role-visible only)", + ) + + class DailyActivityPoint(BaseModel): """Schema for a single day's activity count.""" diff --git a/src/meshcore_hub/common/schemas/routes.py b/src/meshcore_hub/common/schemas/routes.py index 242affc..8864311 100644 --- a/src/meshcore_hub/common/schemas/routes.py +++ b/src/meshcore_hub/common/schemas/routes.py @@ -260,3 +260,38 @@ class RouteHistory(BaseModel): route_id: str days: int data: list[RouteDayQuality] + + +class RouteOverviewEntry(BaseModel): + """One route's current state plus its recent history (dashboard widget).""" + + id: str + from_label: str + to_label: str + visibility: str + enabled: bool + state: Optional[str] = None + quality: Optional[str] = None + matched_count: Optional[int] = None + history: list[RouteDayQuality] = [] + + +class RoutesOverview(BaseModel): + """Aggregate route-fleet snapshot for the dashboard. + + - ``by_state`` buckets routes by their current ``state`` (clear / + marginal / failing / no_coverage / disabled) so a single stacked + bar can render fleet health. + - ``routes`` carries per-route 7-day history for the trend line and + the per-route strip grid. The frontend slices the top N by + ``matched_count`` for display. + """ + + days: int + by_state: list["BreakdownBucket"] + routes: list[RouteOverviewEntry] + + +from meshcore_hub.common.schemas.messages import BreakdownBucket # noqa: E402 + +RoutesOverview.model_rebuild() diff --git a/src/meshcore_hub/web/static/css/app.css b/src/meshcore_hub/web/static/css/app.css index 7cef40b..afc28dd 100644 --- a/src/meshcore_hub/web/static/css/app.css +++ b/src/meshcore_hub/web/static/css/app.css @@ -86,6 +86,8 @@ .nav-icon-dashboard { color: var(--color-dashboard); } .nav-icon-nodes { color: var(--color-nodes); } .nav-icon-adverts { color: var(--color-adverts); } +.nav-icon-channels { color: var(--color-channels); } +.nav-icon-routes { color: var(--color-routes); } .nav-icon-messages { color: var(--color-messages); } .nav-icon-packets { color: var(--color-packets); } .nav-icon-map { color: var(--color-map); } @@ -95,6 +97,8 @@ .navbar .menu li:has(.nav-icon-dashboard) { --nav-color: var(--color-dashboard); } .navbar .menu li:has(.nav-icon-nodes) { --nav-color: var(--color-nodes); } .navbar .menu li:has(.nav-icon-adverts) { --nav-color: var(--color-adverts); } +.navbar .menu li:has(.nav-icon-channels) { --nav-color: var(--color-channels); } +.navbar .menu li:has(.nav-icon-routes) { --nav-color: var(--color-routes); } .navbar .menu li:has(.nav-icon-messages) { --nav-color: var(--color-messages); } .navbar .menu li:has(.nav-icon-packets) { --nav-color: var(--color-packets); } .navbar .menu li:has(.nav-icon-map) { --nav-color: var(--color-map); } @@ -125,6 +129,16 @@ border-left: 5px solid color-mix(in oklch, var(--panel-color, transparent) 55%, transparent); } +/* Route-health strip grid (dashboard widget). + Each cell is a fixed-width colored square; the row flexes so long + route labels truncate while the strip stays compact. */ +.route-health-cell { + width: 14px; + height: 14px; + border-radius: 2px; + display: inline-block; +} + .radio-tile-icon { color: var(--color-radio); } diff --git a/src/meshcore_hub/web/static/js/charts.js b/src/meshcore_hub/web/static/js/charts.js index 71613c9..c0c1fb6 100644 --- a/src/meshcore_hub/web/static/js/charts.js +++ b/src/meshcore_hub/web/static/js/charts.js @@ -42,6 +42,8 @@ const ChartColors = { get messagesFill() { return withAlpha(this.messages, 0.1); }, get packets() { return getCSSColor('--color-packets', 'oklch(0.72 0.17 145)'); }, get packetsFill() { return withAlpha(this.packets, 0.1); }, + get routes() { return getCSSColor('--color-routes', 'oklch(0.72 0.17 30)'); }, + get routesFill() { return withAlpha(this.routes, 0.1); }, // Neutral grays (not page-specific) grid: 'oklch(0.4 0 0 / 0.2)', @@ -322,9 +324,142 @@ function createStackedBarChart(canvasId, buckets, colors) { }); } +/** + * Create a multi-line route-status trend chart for the dashboard. + * + * Each route becomes one line plotted on a 3-tier categorical Y axis + * (``failing`` → ``marginal`` → ``clear``, bottom to top). The line's + * color reflects the route's CURRENT quality (its latest evaluation), + * so multiple routes in the same health band share a color — the chart + * reads as a fleet-health overview rather than per-route identity. + * Hover tooltips still show the route label, tier, and matched_count. + * + * Input is the ``routes`` array from ``GET /dashboard/routes-overview``. + * The top ``maxRoutes`` routes by current ``matched_count`` are drawn; + * the rest are dropped silently (summing quality tiers is meaningless). + * + * Quality → tier mapping (per the merged-3-tier design): + * ``clear`` → clear + * ``marginal`` → marginal + * anything else → failing (covers ``failing``, ``unknown``, + * ``no_coverage``, ``disabled``, null) + * + * @param {string} canvasId - ID of the canvas element + * @param {Array|null} routes - Array of RouteOverviewEntry objects + * @param {number} [maxRoutes=6] - Top-N routes drawn distinctly + * @returns {Chart|null} + */ +function createRoutesTrendChart(canvasId, routes, maxRoutes) { + var ctx = document.getElementById(canvasId); + if (!ctx || !routes || routes.length === 0) return null; + + maxRoutes = maxRoutes || 6; + + // Bottom-to-top tier order on the categorical Y axis. + var tierOrder = ['failing', 'marginal', 'clear']; + + function qualityToTier(q) { + if (q === 'clear') return 'clear'; + if (q === 'marginal') return 'marginal'; + return 'failing'; + } + + function tierColor(tier) { + return ChartColors.quality[tier] || ChartColors.quality.failing; + } + + // Mean tier over the displayed window. Maps the 3-tier space onto a + // 0/1/2 numeric scale (failing < marginal < clear), averages, then + // buckets back: >=1.5 → clear, >=0.75 → marginal, else failing. + // Empty history falls through to failing (matches qualityToTier's + // default for unknown / null quality). + function averageTier(history) { + if (!history || history.length === 0) return 'failing'; + var sum = 0; + for (var i = 0; i < history.length; i++) { + var tier = qualityToTier(history[i].quality); + sum += (tier === 'clear' ? 2 : tier === 'marginal' ? 1 : 0); + } + var mean = sum / history.length; + if (mean >= 1.5) return 'clear'; + if (mean >= 0.75) return 'marginal'; + return 'failing'; + } + + // Sort by current matched_count desc; routes with null matched_count + // (disabled / never evaluated) sort to the end. + var sorted = routes.slice().sort(function(a, b) { + var am = a.matched_count || 0; + var bm = b.matched_count || 0; + return bm - am; + }); + var top = sorted.slice(0, maxRoutes); + + // Use the longest history as the X-axis label source (all routes + // share the same window in practice, but be defensive). + var labels = []; + for (var i = 0; i < top.length; i++) { + if (top[i].history && top[i].history.length > labels.length) { + labels = formatDateLabels(top[i].history); + } + } + if (labels.length === 0) return null; + + var datasets = top.map(function(entry) { + var history = entry.history || []; + var avgTier = averageTier(history); + return { + label: entry.from_label + ' \u2192 ' + entry.to_label, + data: history.map(function(d) { return qualityToTier(d.quality); }), + borderColor: tierColor(avgTier), + backgroundColor: 'transparent', + fill: false, + tension: 0.3, + cubicInterpolationMode: 'monotone', + pointRadius: 2, + pointHoverRadius: 5, + spanGaps: true, + _matched: history.map(function(d) { return d.matched_count || 0; }) + }; + }); + + var opts = createChartOptions(false); + // Replace the default numeric Y axis with a 3-tier categorical axis. + opts.scales.y = { + type: 'category', + labels: tierOrder, + reverse: true, + grid: { color: ChartColors.grid }, + ticks: { + color: ChartColors.text, + callback: function(_value, index) { + var tier = tierOrder[index]; + return (window.t && window.t('routes.quality_' + tier)) || tier; + } + } + }; + // The default tooltip formatter calls formatNumber(ctx.parsed.y), + // which is wrong for categorical string values; emit tier + matched. + opts.plugins.tooltip.callbacks = { + title: function(items) { return items[0].label; }, + label: function(ctx) { + var tier = tierOrder[ctx.parsed.y] || 'failing'; + var tierLabel = (window.t && window.t('routes.quality_' + tier)) || tier; + var matched = (ctx.dataset._matched && ctx.dataset._matched[ctx.dataIndex]) || 0; + return ctx.dataset.label + ': ' + tierLabel + ' (' + matched + ')'; + } + }; + + return new Chart(ctx, { + type: 'line', + data: { labels: labels, datasets: datasets }, + options: opts + }); +} + /** * Initialize dashboard charts (nodes, advertisements, messages, packets, - * plus optional packet-breakdown stacked bars). + * plus optional packet-breakdown stacked bars and routes overview). * Pass null for any data parameter to skip that chart. * @param {Object|null} nodeData - Node count data, or null to skip * @param {Object|null} advertData - Advertisement data, or null to skip @@ -332,8 +467,9 @@ function createStackedBarChart(canvasId, buckets, colors) { * @param {Object|null} packetData - Raw-packet trend data, or null to skip * @param {Array|null} [eventTypeData] - Packet event-type breakdown buckets * @param {Array|null} [pathWidthData] - Packet path-width breakdown buckets + * @param {Array|null} [routesData] - Routes overview ``routes`` array */ -function initDashboardCharts(nodeData, advertData, messageData, packetData, eventTypeData, pathWidthData) { +function initDashboardCharts(nodeData, advertData, messageData, packetData, eventTypeData, pathWidthData, routesData) { if (nodeData) { createLineChart( 'nodeChart', @@ -393,6 +529,10 @@ function initDashboardCharts(nodeData, advertData, messageData, packetData, even ChartColors.breakdown.slice(0, 3) ); } + + if (routesData && routesData.length > 0) { + createRoutesTrendChart('routesTrendChart', routesData); + } } /** diff --git a/src/meshcore_hub/web/static/js/spa/app.js b/src/meshcore_hub/web/static/js/spa/app.js index f0f1575..d617122 100644 --- a/src/meshcore_hub/web/static/js/spa/app.js +++ b/src/meshcore_hub/web/static/js/spa/app.js @@ -231,11 +231,11 @@ function renderMobileNav(config) { if (features.advertisements !== false) { items.push(html`
  • ${iconAdvertisements('h-5 w-5 nav-icon-adverts')} ${t('entities.advertisements')}
    • `); } - if (features.channels !== false) { - items.push(html`
  • ${iconChannel('h-5 w-5')} ${t('entities.channels')}
    • `); - } if (features.routes !== false) { - items.push(html`
  • ${iconPath('h-5 w-5')} ${t('entities.routes')}
    • `); + items.push(html`
  • ${iconPath('h-5 w-5 nav-icon-routes')} ${t('entities.routes')}
    • `); + } + if (features.channels !== false) { + items.push(html`
  • ${iconChannel('h-5 w-5 nav-icon-channels')} ${t('entities.channels')}
    • `); } if (features.messages !== false) { items.push(html`
  • ${iconMessages('h-5 w-5 nav-icon-messages')} ${t('entities.messages')}
    • `); diff --git a/src/meshcore_hub/web/static/js/spa/components.js b/src/meshcore_hub/web/static/js/spa/components.js index c2dcc31..1232bd9 100644 --- a/src/meshcore_hub/web/static/js/spa/components.js +++ b/src/meshcore_hub/web/static/js/spa/components.js @@ -659,10 +659,12 @@ export function observerFilterBadges({ areas, disabled, onToggle, extraClass = ' const title = enabled ? t('common.filter_observer_disable') : t('common.filter_observer_enable'); + const emoji = extractFirstEmoji(n._displayName); + const label = emoji ? (n._displayName.replace(emoji, '').trim() || n._displayName) : n._displayName; return html``; + @click=${() => onToggle(n.public_key)}>${emoji ? html`${emoji}` : nothing}${label}`; })} `; } diff --git a/src/meshcore_hub/web/static/js/spa/pages/dashboard.js b/src/meshcore_hub/web/static/js/spa/pages/dashboard.js index 7bc3608..b884e45 100644 --- a/src/meshcore_hub/web/static/js/spa/pages/dashboard.js +++ b/src/meshcore_hub/web/static/js/spa/pages/dashboard.js @@ -119,12 +119,62 @@ function gridCols(count) { return ''; } -function renderChartCards({ showNodes, showAdverts, showMessages, showPackets, stats, packetBreakdown }) { +function renderRoutesHealth(routes) { + if (!routes || routes.length === 0) { + return html`

    ${t('dashboard.routes_empty')}

    `; + } + // Top 6 by current matched_count, mirroring the trend chart cap so the + // two widgets surface the same routes. + const sorted = routes.slice().sort((a, b) => (b.matched_count || 0) - (a.matched_count || 0)); + const maxRows = 6; + const visible = sorted.slice(0, maxRows); + const hidden = sorted.length - visible.length; + + const colorFor = (q) => { + // Mirrors ChartColors.quality in charts.js but reads CSS vars so the + // strips stay in sync with the legend. + const map = { + clear: 'oklch(0.72 0.17 145)', + marginal: 'oklch(0.75 0.18 85)', + failing: 'oklch(0.62 0.24 25)', + no_coverage: 'oklch(0.65 0.15 250)', + disabled: 'oklch(0.55 0 0)', + }; + return map[q] || map.no_coverage; + }; + const labelFor = (q) => t('routes.quality_' + (q || 'unknown')); + + const rows = visible.map(r => { + const cells = (r.history || []).map(d => html` +
    `); + const current = r.quality || (r.enabled ? 'no_coverage' : 'disabled'); + return html`
    + + ${r.from_label} \u2192 ${r.to_label} + +
    ${cells}
    + +
    `; + }); + + return html`
    + ${rows} + ${hidden > 0 ? html`

    ${t('dashboard.routes_more', { count: hidden })}

    ` : nothing} +
    `; +} + +function renderChartCards({ showNodes, showAdverts, showMessages, showPackets, showRoutes, stats, packetBreakdown, routesOverview }) { const visibleCount = (showNodes ? 1 : 0) + (showAdverts ? 1 : 0) + (showMessages ? 1 : 0) + (showPackets ? 1 : 0); if (visibleCount === 0) return nothing; const eventTypeTotal = packetBreakdown?.by_event_type?.reduce((s, b) => s + b.count, 0) ?? 0; const pathWidthTotal = packetBreakdown?.by_path_width?.reduce((s, b) => s + b.count, 0) ?? 0; + const hasRoutes = !!(routesOverview && routesOverview.routes && routesOverview.routes.length); return html`
    @@ -213,8 +263,9 @@ function renderChartCards({ showNodes, showAdverts, showMessages, showPackets, s
    ` : nothing} -${showPackets ? html` -
    +${(showPackets || (showRoutes && hasRoutes)) ? html` +
    + ${showPackets ? html`
    @@ -233,8 +284,9 @@ ${showPackets ? html`
    -
    +
    ` : nothing} + ${showPackets ? html`
    @@ -253,7 +305,39 @@ ${showPackets ? html`
    -
    +
    ` : nothing} + + ${(showRoutes && hasRoutes) ? html` +
    +
    +
    +
    +

    + ${t('dashboard.route_health')} +

    +

    ${t('time.last_7_days')}

    +
    +
    + ${renderRoutesHealth(routesOverview.routes)} +
    +
    ` : nothing} + + ${(showRoutes && hasRoutes) ? html` +
    +
    +
    +
    +

    + ${t('dashboard.routes_trend')} +

    +

    ${t('time.routes_over_last_n_days', { n: routesOverview.days })}

    +
    +
    +
    + +
    +
    +
    ` : nothing} ` : nothing}`; } @@ -267,14 +351,17 @@ export async function render(container, params, router) { const showAdverts = features.advertisements !== false; const showMessages = features.messages !== false; const showPackets = features.packets !== false; + const showRoutes = features.routes !== false; - const [stats, advertActivity, messageActivity, nodeCount, packetActivity, packetBreakdown, channelsData] = await Promise.all([ + const [stats, recentActivity, advertActivity, messageActivity, nodeCount, packetActivity, packetBreakdown, routesOverview, channelsData] = await Promise.all([ apiGet('/api/v1/dashboard/stats', {}, { signal }), + apiGet('/api/v1/dashboard/recent-activity', {}, { signal }), apiGet('/api/v1/dashboard/activity', { days: 7 }, { signal }), apiGet('/api/v1/dashboard/message-activity', { days: 7 }, { signal }), apiGet('/api/v1/dashboard/node-count', { days: 7 }, { signal }), apiGet('/api/v1/dashboard/packet-activity', { days: 7 }, { signal }), apiGet('/api/v1/dashboard/packet-breakdown', { days: 7 }, { signal }), + showRoutes ? apiGet('/api/v1/dashboard/routes-overview', { days: 7 }, { signal }) : Promise.resolve(null), apiGet('/api/v1/channels', {}, { signal }), ]); channelLabels = new Map([ @@ -294,7 +381,7 @@ export async function render(container, params, router) { ${(showNodes || showAdverts || showMessages || showPackets) ? html` -${renderChartCards({ showNodes, showAdverts, showMessages, showPackets, stats, packetBreakdown })}` : nothing} +${renderChartCards({ showNodes, showAdverts, showMessages, showPackets, showRoutes, stats, packetBreakdown, routesOverview })}` : nothing} ${bottomCount > 0 ? html`
    @@ -305,11 +392,11 @@ ${bottomCount > 0 ? html` ${iconAdvertisements('h-6 w-6')} ${t('common.recent_entity', { entity: t('entities.advertisements') })} - ${renderRecentAds(stats.recent_advertisements)} + ${renderRecentAds(recentActivity.recent_advertisements)}
    ` : nothing} - ${showMessages ? renderChannelMessages(stats.channel_messages, channelLabels) : nothing} + ${showMessages ? renderChannelMessages(recentActivity.channel_messages, channelLabels) : nothing} ` : nothing}`, container); window.initDashboardCharts( @@ -319,9 +406,10 @@ ${bottomCount > 0 ? html` showPackets ? packetActivity : null, showPackets ? packetBreakdown.by_event_type : null, showPackets ? packetBreakdown.by_path_width : null, + (showRoutes && routesOverview && routesOverview.routes) ? routesOverview.routes : null, ); - const chartIds = ['nodeChart', 'advertChart', 'messageChart', 'packetChart', 'packetEventTypeChart', 'packetPathWidthChart']; + const chartIds = ['nodeChart', 'advertChart', 'messageChart', 'packetChart', 'packetEventTypeChart', 'packetPathWidthChart', 'routesTrendChart']; return () => { chartIds.forEach(id => { const canvas = document.getElementById(id); diff --git a/src/meshcore_hub/web/static/js/spa/pages/home.js b/src/meshcore_hub/web/static/js/spa/pages/home.js index d5ef873..17cdfc1 100644 --- a/src/meshcore_hub/web/static/js/spa/pages/home.js +++ b/src/meshcore_hub/web/static/js/spa/pages/home.js @@ -96,18 +96,18 @@ function renderHeroSection({ networkName, logoUrl, logoInvertLight, networkCity, label: t('entities.advertisements'), colorVar: '--color-adverts', }) : nothing} - ${features.channels !== false ? renderNavCard({ - href: '/channels', - icon: iconChannel('w-full h-full'), - label: t('entities.channels'), - colorVar: '--color-channels', - }) : nothing} ${features.routes !== false ? renderNavCard({ href: '/routes', icon: iconPath('w-full h-full'), label: t('entities.routes'), colorVar: '--color-routes', }) : nothing} + ${features.channels !== false ? renderNavCard({ + href: '/channels', + icon: iconChannel('w-full h-full'), + label: t('entities.channels'), + colorVar: '--color-channels', + }) : nothing} ${features.messages !== false ? renderNavCard({ href: '/messages', icon: iconMessages('w-full h-full'), diff --git a/src/meshcore_hub/web/static/js/spa/pages/routes.js b/src/meshcore_hub/web/static/js/spa/pages/routes.js index 3f2084e..e02b5b2 100644 --- a/src/meshcore_hub/web/static/js/spa/pages/routes.js +++ b/src/meshcore_hub/web/static/js/spa/pages/routes.js @@ -121,7 +121,6 @@ function renderRouteCard(route, { isAdmin, onDelete, onEdit, detail, navigate, p const label = qualityLabel(q, route.enabled); const dot = qualityDot(q, route.enabled); const tip = diagnosisText(route); - const arrow = route.reversible !== false ? '\u2194' : '\u2192'; const badge = tip ? html`${dot} ${label}` : html`${dot} ${label}`; @@ -158,7 +157,6 @@ function renderRouteCard(route, { isAdmin, onDelete, onEdit, detail, navigate, p ${route.description ? html`

    ${route.description}

    ` : nothing}
    - ${arrow} ${badge}
    @@ -181,7 +179,7 @@ function renderDetailContent(route, detail, { navigate, packetsEnabled, history ${history.data && history.data.length > 0 ? html`
    - ${history.data.map(d => html`${new Date(d.date + 'T00:00:00').toLocaleDateString(undefined, { day: '2-digit', month: '2-digit' })}`)} + ${history.data.map((d, i) => html`${i === history.data.length - 1 ? t('routes.last_n_hours', { n: route.window_hours }) : new Date(d.date + 'T00:00:00').toLocaleDateString(undefined, { day: '2-digit', month: '2-digit' })}`)}
    ` : nothing} ` : nothing; diff --git a/src/meshcore_hub/web/static/locales/en.json b/src/meshcore_hub/web/static/locales/en.json index e42b015..9b53646 100644 --- a/src/meshcore_hub/web/static/locales/en.json +++ b/src/meshcore_hub/web/static/locales/en.json @@ -138,7 +138,8 @@ "last_7_days": "Last 7 days", "per_day_last_7_days": "Per day (last 7 days)", "over_time_last_7_days": "Over time (last 7 days)", - "activity_per_day_last_7_days": "Activity per day (last 7 days)" + "activity_per_day_last_7_days": "Activity per day (last 7 days)", + "routes_over_last_n_days": "Matched packets per route (last {{n}} days)" }, "node_types": { "chat": "Chat", @@ -161,7 +162,11 @@ "dashboard": { "all_discovered_nodes": "All discovered nodes", "recent_channel_messages": "Recent Channel Messages", - "channel": "Channel {{number}}" + "channel": "Channel {{number}}", + "routes_trend": "Route Trends", + "route_health": "Route Health", + "routes_more": "+{{count}} more routes", + "routes_empty": "No routes configured yet." }, "nodes": { "scan_to_add": "Scan to add as contact", @@ -346,7 +351,9 @@ "quality_no_coverage": "No Coverage", "quality_unknown": "Unknown", "recent_packets": "Recent Packets", - "min_nodes_error": "At least 2 path nodes are required." + "last_n_hours": "Last {{n}}h", + "min_nodes_error": "At least 2 path nodes are required.", + "other_routes": "Other routes" }, "not_found": { "description": "The page you're looking for doesn't exist or has been moved." diff --git a/src/meshcore_hub/web/static/locales/nl.json b/src/meshcore_hub/web/static/locales/nl.json index 9bbdcdd..f7622be 100644 --- a/src/meshcore_hub/web/static/locales/nl.json +++ b/src/meshcore_hub/web/static/locales/nl.json @@ -128,7 +128,8 @@ "last_7_days": "Laatste 7 dagen", "per_day_last_7_days": "Per dag (laatste 7 dagen)", "over_time_last_7_days": "In de tijd (laatste 7 dagen)", - "activity_per_day_last_7_days": "Activiteit per dag (laatste 7 dagen)" + "activity_per_day_last_7_days": "Activiteit per dag (laatste 7 dagen)", + "routes_over_last_n_days": "Overeenkomende pakketten per route (laatste {{n}} dagen)" }, "node_types": { "chat": "Chat", @@ -150,7 +151,11 @@ "dashboard": { "all_discovered_nodes": "Alle ontdekte knooppunten", "recent_channel_messages": "Recente kanaalberichten", - "channel": "Kanaal {{number}}" + "channel": "Kanaal {{number}}", + "routes_trend": "Route-trends", + "route_health": "Route-status", + "routes_more": "+{{count}} routes meer", + "routes_empty": "Nog geen routes geconfigureerd." }, "nodes": { "scan_to_add": "Scan om als contact toe te voegen" @@ -268,7 +273,9 @@ "quality_no_coverage": "Geen dekking", "quality_unknown": "Onbekend", "recent_packets": "Recente packets", - "min_nodes_error": "Minimaal 2 padknooppunten zijn vereist." + "last_n_hours": "Laatste {{n}}u", + "min_nodes_error": "Minimaal 2 padknooppunten zijn vereist.", + "other_routes": "Overige routes" }, "not_found": { "description": "De pagina die u zoekt bestaat niet of is verplaatst." diff --git a/src/meshcore_hub/web/templates/spa.html b/src/meshcore_hub/web/templates/spa.html index 8af3ba6..2cb5655 100644 --- a/src/meshcore_hub/web/templates/spa.html +++ b/src/meshcore_hub/web/templates/spa.html @@ -70,12 +70,12 @@ {% if features.advertisements %}
  • {{ t('entities.advertisements') }}
    • {% endif %} - {% if features.channels %} -
  • {{ t('entities.channels') }}
    • - {% endif %} {% if features.routes %}
  • {{ t('entities.routes') }}
    • {% endif %} + {% if features.channels %} +
  • {{ t('entities.channels') }}
    • + {% endif %} {% if features.messages %}
  • {{ t('entities.messages') }}
    • {% endif %} diff --git a/tests/test_api/test_cache.py b/tests/test_api/test_cache.py index 99a94d9..9393675 100644 --- a/tests/test_api/test_cache.py +++ b/tests/test_api/test_cache.py @@ -436,9 +436,9 @@ class TestCachedDecorator: mock_cache.get.return_value = None app.state.redis_cache = mock_cache app.state.redis_cache_ttl = 30 - app.state.redis_cache_ttl_route_detail = 90 + app.state.redis_cache_ttl_dashboard = 90 - @cached("routes/{id}", ttl_setting="redis_cache_ttl_route_detail") + @cached("routes/{id}", ttl_setting="redis_cache_ttl_dashboard") async def handler(request: Request): return {"id": "abc", "matches": []} @@ -807,9 +807,9 @@ class TestCachedEtag: app = FastAPI() app.state.redis_cache = NullCache() app.state.redis_cache_ttl = 30 - app.state.redis_cache_ttl_route_detail = 90 + app.state.redis_cache_ttl_dashboard = 90 - @cached("routes/{id}", ttl_setting="redis_cache_ttl_route_detail") + @cached("routes/{id}", ttl_setting="redis_cache_ttl_dashboard") async def handler(request: Request): return {"id": "abc"} @@ -898,7 +898,7 @@ class TestCacheControlMiddleware: mock_cache.get.return_value = None client_no_auth.app.state.redis_cache = mock_cache client_no_auth.app.state.redis_cache_ttl = 30 - client_no_auth.app.state.redis_cache_ttl_route_detail = 300 + client_no_auth.app.state.redis_cache_ttl_dashboard = 300 # Hit the route detail endpoint with a fake id. The endpoint returns # 404 but still flows through the @cached decorator; we only care # that the TTL is NOT surfaced as an HTTP header. @@ -1176,8 +1176,6 @@ class TestCliRedis: "60", "--redis-cache-ttl-dashboard", "120", - "--redis-cache-ttl-route-detail", - "90", "--no-api-cache-control", ], catch_exceptions=False, @@ -1191,7 +1189,6 @@ class TestCliRedis: assert call_kwargs["redis_key_prefix"] == "pre" assert call_kwargs["redis_cache_ttl"] == 60 assert call_kwargs["redis_cache_ttl_dashboard"] == 120 - assert call_kwargs["redis_cache_ttl_route_detail"] == 90 assert call_kwargs["api_cache_control_enabled"] is False def test_api_cache_control_enabled_default(self): @@ -1214,10 +1211,13 @@ class TestCliRedis: assert call_kwargs["api_cache_control_enabled"] is True def test_redis_cache_ttl_dashboard_default(self): - """--redis-cache-ttl-dashboard defaults to 300 (raised from 30). + """--redis-cache-ttl-dashboard defaults to 3600 (1 hour). - Covers /dashboard/* endpoints and /routes/{id}/history. Trend data - tolerates longer staleness than the original 30s default. + Covers /dashboard/* endpoints, /routes/{id} detail, and + /routes/{id}/history. Trend/aggregation data tolerates much longer + staleness than the default 30 s TTL — Recent Adverts / Recent + Channel Messages widgets were split into /dashboard/recent-activity + so they can stay on the short default TTL independently. """ from click.testing import CliRunner @@ -1234,7 +1234,7 @@ class TestCliRedis: mock_create_app.return_value = MagicMock() runner.invoke(api, catch_exceptions=False) call_kwargs = mock_create_app.call_args[1] - assert call_kwargs["redis_cache_ttl_dashboard"] == 300 + assert call_kwargs["redis_cache_ttl_dashboard"] == 3600 class TestKeyBuilders: @@ -1363,8 +1363,12 @@ class TestCacheInvalidationHelpers: cache = MagicMock() invalidate_routes(_make_request_with_cache(cache)) - # Single prefix covers /routes, /routes/{id}, /routes/{id}/history - cache.delete.assert_called_once_with("/api/v1/routes") + # Two prefixes: /api/v1/routes covers the list, detail, and history + # endpoints (URL-path key_builder); dashboard/routes-overview covers + # the dashboard aggregate (endpoint-name key namespace). + assert cache.delete.call_count == 2 + cache.delete.assert_any_call("/api/v1/routes") + cache.delete.assert_any_call("dashboard/routes-overview") def test_invalidate_nodes_drops_endpoint_name_prefix(self): from meshcore_hub.api.cache_invalidation import invalidate_nodes @@ -1438,7 +1442,6 @@ class TestMutationInvalidationIntegration: client.app.state.redis_cache = mock_cache client.app.state.redis_cache_ttl = 30 client.app.state.redis_cache_ttl_dashboard = 300 - client.app.state.redis_cache_ttl_route_detail = 300 return mock_cache # --- Channels --------------------------------------------------------- @@ -1495,6 +1498,10 @@ class TestMutationInvalidationIntegration: ) assert resp.status_code == 201 mock_cache.delete.assert_any_call("/api/v1/routes") + # Dashboard routes-overview embeds per-route state, so it must be + # dropped too — otherwise the new route stays invisible on the + # dashboard until TTL expiry. + mock_cache.delete.assert_any_call("dashboard/routes-overview") def test_update_route_invalidates_routes(self, client_no_auth, api_db_session): from meshcore_hub.common.models import Node, Route, RouteNode @@ -1524,6 +1531,7 @@ class TestMutationInvalidationIntegration: ) assert resp.status_code == 200 mock_cache.delete.assert_any_call("/api/v1/routes") + mock_cache.delete.assert_any_call("dashboard/routes-overview") def test_delete_route_invalidates_routes(self, client_no_auth, api_db_session): from meshcore_hub.common.models import Node, Route, RouteNode @@ -1552,6 +1560,7 @@ class TestMutationInvalidationIntegration: ) assert resp.status_code == 204 mock_cache.delete.assert_any_call("/api/v1/routes") + mock_cache.delete.assert_any_call("dashboard/routes-overview") # --- User profiles ---------------------------------------------------- diff --git a/tests/test_api/test_dashboard.py b/tests/test_api/test_dashboard.py index 3f261c5..77e3554 100644 --- a/tests/test_api/test_dashboard.py +++ b/tests/test_api/test_dashboard.py @@ -16,6 +16,11 @@ from meshcore_hub.common.models import ( RawPacket, ) from meshcore_hub.common.models import UserProfile +from meshcore_hub.common.models import ( + PacketPathHop, + Route, + RouteNode, +) class TestDateBucketKey: @@ -59,6 +64,9 @@ class TestDashboardStats: assert data["total_packets"] == 0 assert data["packets_7d"] == 0 assert data["channel_message_counts"] == {} + # recent_advertisements / channel_messages moved to /recent-activity + assert "recent_advertisements" not in data + assert "channel_messages" not in data def test_get_stats_with_data( self, client_no_auth, sample_node, sample_message, sample_advertisement @@ -838,7 +846,7 @@ class TestDashboardFloodOnlyFilter: api_db_session.add_all([direct_ad, flood_ad]) api_db_session.commit() - response = client_no_auth.get("/api/v1/dashboard/stats") + response = client_no_auth.get("/api/v1/dashboard/recent-activity") assert response.status_code == 200 data = response.json() assert len(data["recent_advertisements"]) == 1 @@ -874,56 +882,62 @@ class TestDashboardFloodOnlyFilter: assert total_count == 1 +@pytest.fixture +def channels_with_messages(api_db_session): + """Create public and admin channels with messages. + + Module-level so both TestDashboardChannelVisibility (channel_message_counts + on /dashboard/stats) and TestDashboardRecentActivity (channel_messages on + /dashboard/recent-activity) can share the same setup. + """ + pub_key = "AABBCCDDEEFF00112233445566778899" + adm_key = "FFEEDDCCBBAA99887766554433221100" + pub_idx = int(Channel.compute_channel_hash(pub_key), 16) + adm_idx = int(Channel.compute_channel_hash(adm_key), 16) + + pub_ch = Channel( + name="CommunityCh", + key_hex=pub_key, + channel_hash=Channel.compute_channel_hash(pub_key), + visibility="community", + enabled=True, + ) + adm_ch = Channel( + name="AdminCh", + key_hex=adm_key, + channel_hash=Channel.compute_channel_hash(adm_key), + visibility="admin", + enabled=True, + ) + api_db_session.add_all([pub_ch, adm_ch]) + + pub_msg = Message( + message_type="channel", + channel_idx=pub_idx, + text="Public message", + received_at=datetime.now(timezone.utc), + ) + adm_msg = Message( + message_type="channel", + channel_idx=adm_idx, + text="Admin message", + received_at=datetime.now(timezone.utc), + ) + direct_msg = Message( + message_type="direct", + pubkey_prefix="abc123", + text="Direct message", + received_at=datetime.now(timezone.utc), + ) + api_db_session.add_all([pub_msg, adm_msg, direct_msg]) + api_db_session.commit() + + return pub_idx, adm_idx + + class TestDashboardChannelVisibility: """Tests for channel visibility filtering on dashboard stats.""" - @pytest.fixture - def channels_with_messages(self, api_db_session): - """Create public and admin channels with messages.""" - pub_key = "AABBCCDDEEFF00112233445566778899" - adm_key = "FFEEDDCCBBAA99887766554433221100" - pub_idx = int(Channel.compute_channel_hash(pub_key), 16) - adm_idx = int(Channel.compute_channel_hash(adm_key), 16) - - pub_ch = Channel( - name="CommunityCh", - key_hex=pub_key, - channel_hash=Channel.compute_channel_hash(pub_key), - visibility="community", - enabled=True, - ) - adm_ch = Channel( - name="AdminCh", - key_hex=adm_key, - channel_hash=Channel.compute_channel_hash(adm_key), - visibility="admin", - enabled=True, - ) - api_db_session.add_all([pub_ch, adm_ch]) - - pub_msg = Message( - message_type="channel", - channel_idx=pub_idx, - text="Public message", - received_at=datetime.now(timezone.utc), - ) - adm_msg = Message( - message_type="channel", - channel_idx=adm_idx, - text="Admin message", - received_at=datetime.now(timezone.utc), - ) - direct_msg = Message( - message_type="direct", - pubkey_prefix="abc123", - text="Direct message", - received_at=datetime.now(timezone.utc), - ) - api_db_session.add_all([pub_msg, adm_msg, direct_msg]) - api_db_session.commit() - - return pub_idx, adm_idx - def test_anonymous_sees_only_community_messages( self, client_no_auth, channels_with_messages ): @@ -1038,7 +1052,7 @@ class TestDashboardChannelVisibility: api_db_session.add(ad) api_db_session.commit() - response = client_no_auth.get("/api/v1/dashboard/stats") + response = client_no_auth.get("/api/v1/dashboard/recent-activity") assert response.status_code == 200 data = response.json() assert len(data["recent_advertisements"]) == 1 @@ -1084,7 +1098,7 @@ class TestDashboardChannelVisibility: api_db_session.add(observer) api_db_session.commit() - response = client_no_auth.get("/api/v1/dashboard/stats") + response = client_no_auth.get("/api/v1/dashboard/recent-activity") assert response.status_code == 200 data = response.json() assert len(data["recent_advertisements"]) == 1 @@ -1118,7 +1132,7 @@ class TestDashboardChannelVisibility: api_db_session.add(ad) api_db_session.commit() - response = client_no_auth.get("/api/v1/dashboard/stats") + response = client_no_auth.get("/api/v1/dashboard/recent-activity") assert response.status_code == 200 data = response.json() assert len(data["recent_advertisements"]) == 1 @@ -1178,6 +1192,76 @@ class TestDashboardChannelVisibility: assert str(adm_idx) not in data["channel_message_counts"] +class TestDashboardRecentActivity: + """Tests for GET /dashboard/recent-activity (split from /dashboard/stats).""" + + def test_recent_activity_empty(self, client_no_auth): + """Recent activity endpoint with empty database returns empty shapes.""" + response = client_no_auth.get("/api/v1/dashboard/recent-activity") + assert response.status_code == 200 + data = response.json() + assert data["recent_advertisements"] == [] + assert data["channel_messages"] == {} + + def test_recent_activity_does_not_return_counts( + self, client_no_auth, channels_with_messages + ): + """The recent-activity endpoint drops channel_message_counts (those + stay on /dashboard/stats as aggregate counts).""" + response = client_no_auth.get("/api/v1/dashboard/recent-activity") + assert response.status_code == 200 + data = response.json() + assert "channel_message_counts" not in data + + def test_recent_activity_channel_messages_visibility( + self, client_no_auth, channels_with_messages + ): + """Anonymous viewers only see community-channel messages.""" + pub_idx, adm_idx = channels_with_messages + response = client_no_auth.get("/api/v1/dashboard/recent-activity") + assert response.status_code == 200 + data = response.json() + assert str(pub_idx) in data["channel_messages"] + assert str(adm_idx) not in data["channel_messages"] + + def test_recent_activity_admin_sees_all_channels( + self, client_no_auth, channels_with_messages + ): + """Admin role sees messages on every channel.""" + pub_idx, adm_idx = channels_with_messages + response = client_no_auth.get( + "/api/v1/dashboard/recent-activity", + headers={"X-User-Roles": "admin"}, + ) + assert response.status_code == 200 + data = response.json() + assert str(pub_idx) in data["channel_messages"] + assert str(adm_idx) in data["channel_messages"] + + def test_recent_activity_caches_at_default_ttl_not_dashboard( + self, client_no_auth, api_db_session + ): + """The recent-activity endpoint uses the default redis_cache_ttl + (not redis_cache_ttl_dashboard), so the Recent Adverts / Recent + Channel Messages widgets stay fresh at the 30 s default while the + aggregate counts on /dashboard/stats cache for an hour.""" + from unittest.mock import MagicMock + + client_no_auth.app.state.redis_cache = MagicMock() + client_no_auth.app.state.redis_cache.get.return_value = None + client_no_auth.app.state.redis_cache_ttl = 30 + client_no_auth.app.state.redis_cache_ttl_dashboard = 3600 + + client_no_auth.get("/api/v1/dashboard/recent-activity") + + # First MISS stores the response at the default TTL (30 s), not + # the dashboard TTL (3600 s) — that's the whole point of the split. + # cache.set is invoked positionally as set(key, envelope, ttl). + client_no_auth.app.state.redis_cache.set.assert_called_once() + args, _ = client_no_auth.app.state.redis_cache.set.call_args + assert args[2] == 30 + + class TestDashboardDateBucketRegression: """Regression tests for the Postgres date-bucket flatline bug. @@ -1281,3 +1365,278 @@ class TestDashboardDateBucketRegression: assert data["data"][seeded_idx]["count"] == 1 if seeded_idx > 0: assert data["data"][seeded_idx - 1]["count"] == 0 + + +# --------------------------------------------------------------------------- +# GET /dashboard/routes-overview +# --------------------------------------------------------------------------- + + +def _make_node_with_pk(session, public_key: str, name: str | None = None) -> Node: + node = Node(public_key=public_key, name=name) + session.add(node) + session.flush() + return node + + +def _make_route_with_nodes( + session, + from_label: str, + to_label: str, + pubkeys: list[str], + visibility: str = "community", + enabled: bool = True, + threshold: int = 3, + clear_bar: int | None = None, + match_width: int = 1, +) -> Route: + nodes = [_make_node_with_pk(session, pk) for pk in pubkeys] + route = Route( + from_label=from_label, + to_label=to_label, + visibility=visibility, + enabled=enabled, + match_width=match_width, + packet_count_threshold=threshold, + clear_threshold=clear_bar, + ) + session.add(route) + session.flush() + for pos, n in enumerate(nodes): + session.add( + RouteNode( + route_id=route.id, + node_id=n.id, + position=pos, + expected_hash=n.public_key[: 2 * match_width].upper(), + ) + ) + session.flush() + return route + + +def _make_reception_for_route( + session, + packet_hash: str, + path_hashes: list[str], + received_at: datetime, +) -> None: + """Insert a RawPacket + PacketPathHop rows for a test reception.""" + from uuid import uuid4 + + rp_id = str(uuid4()) + session.add( + RawPacket( + id=rp_id, + packet_hash=packet_hash, + received_at=received_at, + ) + ) + session.flush() + for pos, nh in enumerate(path_hashes): + session.add( + PacketPathHop( + raw_packet_id=rp_id, + position=pos, + node_hash=nh, + packet_hash=packet_hash, + received_at=received_at, + ) + ) + session.flush() + + +class TestRoutesOverview: + """Tests for GET /dashboard/routes-overview.""" + + def test_empty(self, client_no_auth): + """No routes seeded → empty routes array and by_state.""" + resp = client_no_auth.get("/api/v1/dashboard/routes-overview") + assert resp.status_code == 200 + data = resp.json() + assert data["routes"] == [] + assert data["by_state"] == [] + assert data["days"] == 7 + + def test_default_days_is_7(self, client_no_auth, api_db_session): + resp = client_no_auth.get("/api/v1/dashboard/routes-overview") + assert resp.json()["days"] == 7 + + def test_custom_days(self, client_no_auth, api_db_session): + resp = client_no_auth.get("/api/v1/dashboard/routes-overview?days=3") + assert resp.json()["days"] == 3 + + def test_days_clamped_to_retention(self, client_no_auth, api_db_session): + from meshcore_hub.common.config import get_collector_settings + + retention = get_collector_settings().effective_raw_packet_retention_days + resp = client_no_auth.get( + f"/api/v1/dashboard/routes-overview?days={retention + 30}" + ) + assert resp.json()["days"] == retention + + def test_history_includes_today_segment(self, client_no_auth, api_db_session): + """Each route's history has ``days + 1`` entries (include_today).""" + _make_route_with_nodes(api_db_session, "A", "B", ["a" * 64, "b" * 64]) + api_db_session.commit() + + data = client_no_auth.get("/api/v1/dashboard/routes-overview?days=7").json() + assert len(data["routes"]) == 1 + history = data["routes"][0]["history"] + # 7 calendar days + today segment = 8 entries. + assert len(history) == 8 + # Today's entry has the rolling-window state semantics. + today_entry = history[-1] + assert today_entry["quality"] in { + "clear", + "marginal", + "failing", + "unknown", + } + assert today_entry["state"] in { + "healthy", + "unhealthy", + "no_coverage", + } + + def test_visibility_filter_hides_admin_routes(self, client_no_auth, api_db_session): + """Anonymous users must not see admin-only routes in the overview.""" + _make_route_with_nodes( + api_db_session, + "Public", + "Endpoint", + ["a" * 64, "b" * 64], + visibility="community", + ) + _make_route_with_nodes( + api_db_session, + "Secret", + "Endpoint", + ["c" * 64, "d" * 64], + visibility="admin", + ) + api_db_session.commit() + + # Anonymous: only the community route. + anon = client_no_auth.get("/api/v1/dashboard/routes-overview").json() + labels = [r["from_label"] for r in anon["routes"]] + assert labels == ["Public"] + + # Admin sees both. + admin = client_no_auth.get( + "/api/v1/dashboard/routes-overview", + headers={"X-User-Roles": "admin"}, + ).json() + admin_labels = sorted(r["from_label"] for r in admin["routes"]) + assert admin_labels == ["Public", "Secret"] + + def test_by_state_buckets_current_state(self, client_no_auth, api_db_session): + """``by_state`` counts route current state across the fleet.""" + from meshcore_hub.collector.routes import ( + evaluate_route, + upsert_route_result, + ) + + # Healthy route (3 packets in window, low clear bar) → healthy. + healthy = _make_route_with_nodes( + api_db_session, + "Healthy", + "End", + ["a" * 64, "b" * 64], + threshold=3, + clear_bar=3, + ) + now = datetime.now(timezone.utc) + for i in range(3): + _make_reception_for_route( + api_db_session, + f"pk{i}", + ["AA", "BB"], + now - timedelta(hours=1, seconds=i), + ) + # Disabled route → state=disabled regardless of packets. + _make_route_with_nodes( + api_db_session, + "Disabled", + "End", + ["c" * 64, "e" * 64], + enabled=False, + ) + # Fresh route (no result yet, no packets) → no_coverage fallback. + _make_route_with_nodes( + api_db_session, + "Fresh", + "End", + ["f" * 64, "9" * 64], + ) + # Populate route_result for the healthy route like the evaluator does. + since = now - timedelta(hours=healthy.window_hours) + state, quality, matched = evaluate_route(api_db_session, healthy, since) + upsert_route_result(api_db_session, healthy, state, quality, matched) + api_db_session.commit() + + data = client_no_auth.get("/api/v1/dashboard/routes-overview").json() + state_map = {b["label"]: b["count"] for b in data["by_state"]} + # Healthy route picks up `healthy`; disabled picks up `disabled`; + # fresh route with no route_result falls back to `no_coverage`. + assert state_map.get("healthy", 0) >= 1 + assert state_map.get("disabled", 0) >= 1 + assert state_map.get("no_coverage", 0) >= 1 + + # The healthy route's per-route entry carries the matching state. + healthy_entry = next(r for r in data["routes"] if r["from_label"] == "Healthy") + assert healthy_entry["state"] == "healthy" + assert healthy_entry["quality"] == "clear" + assert healthy_entry["matched_count"] == 3 + + def test_disabled_route_matched_count_is_none(self, client_no_auth, api_db_session): + _make_route_with_nodes( + api_db_session, + "Off", + "End", + ["a" * 64, "b" * 64], + enabled=False, + ) + api_db_session.commit() + + data = client_no_auth.get("/api/v1/dashboard/routes-overview").json() + route = data["routes"][0] + assert route["state"] == "disabled" + assert route["matched_count"] is None + + def test_cache_key_is_role_scoped(self, client_no_auth, api_db_session): + """Different roles get independent cache entries (sanity: both + return 200, response differs when visibility-filtered).""" + from unittest.mock import MagicMock + + _make_route_with_nodes( + api_db_session, + "AdminOnly", + "End", + ["a" * 64, "b" * 64], + visibility="admin", + ) + api_db_session.commit() + + # Mock cache that records every set() call's key. + keys_seen: list[str] = [] + mock = MagicMock() + mock.get.return_value = None + + def _set(key, value, ttl): + keys_seen.append(key) + + mock.set.side_effect = _set + client_no_auth.app.state.redis_cache = mock + client_no_auth.app.state.redis_cache_ttl = 30 + + client_no_auth.get("/api/v1/dashboard/routes-overview") + client_no_auth.get( + "/api/v1/dashboard/routes-overview", + headers={"X-User-Roles": "admin"}, + ) + + # Two cache fills, role differs between them. + assert len(keys_seen) == 2 + assert any("anonymous" in k for k in keys_seen) + assert any("admin" in k for k in keys_seen) diff --git a/tests/test_api/test_routes.py b/tests/test_api/test_routes.py index 209e4bf..8668ec2 100644 --- a/tests/test_api/test_routes.py +++ b/tests/test_api/test_routes.py @@ -270,7 +270,7 @@ class TestGetRouteDetail: mock_cache = MagicMock() mock_cache.get.return_value = None client_no_auth.app.state.redis_cache = mock_cache - client_no_auth.app.state.redis_cache_ttl_route_detail = 90 + client_no_auth.app.state.redis_cache_ttl_dashboard = 90 resp = client_no_auth.get(f"/api/v1/routes/{route.id}") assert resp.status_code == 200 @@ -315,7 +315,7 @@ class TestGetRouteDetail: return True client_no_auth.app.state.redis_cache = _FakeCache() - client_no_auth.app.state.redis_cache_ttl_route_detail = 60 + client_no_auth.app.state.redis_cache_ttl_dashboard = 60 first = client_no_auth.get(f"/api/v1/routes/{route.id}") assert first.status_code == 200 @@ -687,6 +687,63 @@ class TestRouteHistory: # retention=2 → days clamped to 2 → 3 entries (days + today) assert len(data["data"]) == 3 + def test_today_segment_matches_route_result_badge( + self, client_no_auth, api_db_session + ): + """The rightmost history segment must match route_result (the badge). + + Regression for the rolling-window/calendar-day mismatch: packets + placed yesterday evening are inside the rolling 24h window used by + the badge but outside today's UTC calendar bucket. After this fix, + the history endpoint's today segment must reflect the same + matched_count/quality/state as route_result. + """ + from datetime import timedelta + + from meshcore_hub.collector.routes import ( + evaluate_route, + upsert_route_result, + ) + + route = _make_route_with_nodes( + api_db_session, "A", "B", ["aa" + "0" * 62, "bb" + "0" * 62] + ) + api_db_session.commit() + + # Place 3 matching packets 6h ago — within the 24h rolling window. + # If "now" is early in the UTC day, this may land in yesterday's + # calendar bucket, which is exactly the scenario we're fixing. + now = datetime.now(timezone.utc) + for i in range(3): + _make_reception( + api_db_session, + None, + f"badge{i}", + ["AA", "BB"], + received_at=now - timedelta(hours=6, seconds=i), + ) + api_db_session.commit() + + # Populate route_result the same way the background evaluator does. + rolling_start = now - timedelta(hours=route.window_hours) + state, quality, matched = evaluate_route(api_db_session, route, rolling_start) + upsert_route_result(api_db_session, route, state, quality, matched) + api_db_session.commit() + api_db_session.refresh(route) + + badge = route.route_result + assert badge is not None, "fixture: route_result must be populated" + + # Bypass cache to read fresh state. + client_no_auth.app.dependency_overrides = {} + + resp = client_no_auth.get(f"/api/v1/routes/{route.id}/history?days=3") + assert resp.status_code == 200 + today_segment = resp.json()["data"][-1] + assert today_segment["matched_count"] == badge.matched_count + assert today_segment["quality"] == badge.quality + assert today_segment["state"] == badge.state + # --------------------------------------------------------------------------- # Additional coverage: update fields, observers, hidden detail, preview guards diff --git a/tests/test_collector/test_routes.py b/tests/test_collector/test_routes.py index 3bb2e9c..c0d09ec 100644 --- a/tests/test_collector/test_routes.py +++ b/tests/test_collector/test_routes.py @@ -999,6 +999,91 @@ class TestEvaluateRouteHistory: assert by_date[clear_date][0] == RouteQuality.CLEAR.value assert by_date[marginal_date][0] == RouteQuality.MARGINAL.value + def test_today_segment_uses_rolling_window(self, db_session): + """include_today segment matches evaluate_route(now - window_hours), + not the calendar-day [00:00 UTC, now] bucket. + + Packets placed yesterday evening fall outside today's UTC calendar + bucket but inside the rolling 24h window — they must surface in the + appended today segment so the chart matches the badge. + """ + node_a = _make_node(db_session, "aa" + "0" * 62) + node_b = _make_node(db_session, "bb" + "0" * 62) + route = _make_route( + db_session, + "R1", + [node_a, node_b], + threshold=3, + clear_bar=3, + window_hours=24, + ) + + # Pick "now" early in the UTC day so yesterday evening is inside the + # 24h rolling window but outside today's calendar bucket. + now = datetime(2026, 7, 15, 3, 0, 0, tzinfo=timezone.utc) + # 3 packets at 21:00 yesterday UTC — within rolling window, + # before today_midnight (2026-07-15 00:00 UTC). + for i in range(3): + _make_reception( + db_session, + None, + f"rolling{i}", + ["AA", "BB"], + received_at=now - timedelta(hours=6, seconds=i), + ) + db_session.commit() + + results = evaluate_route_history( + db_session, route, days=3, include_today=True, now=now + ) + + assert len(results) == 4 + today_entry = results[-1] + assert today_entry[0] == now.date() + # Badge-aligned: 3 matches in the rolling window → HEALTHY/CLEAR + assert today_entry[1] == RouteQuality.CLEAR.value + assert today_entry[2] == RouteState.HEALTHY.value + assert today_entry[3] == 3 + + # Cross-check: the today segment must agree with evaluate_route + # over the same rolling window. + rolling_start = now - timedelta(hours=route.window_hours) + state, quality, matched = evaluate_route(db_session, route, rolling_start) + assert (today_entry[1], today_entry[2], today_entry[3]) == ( + quality, + state, + matched, + ) + + def test_today_segment_excludes_packets_outside_window(self, db_session): + """Packets older than window_hours must not count in today's segment.""" + node_a = _make_node(db_session, "aa" + "0" * 62) + node_b = _make_node(db_session, "bb" + "0" * 62) + # window_hours=6 — only packets in the last 6h count. + route = _make_route( + db_session, "R1", [node_a, node_b], threshold=1, window_hours=6 + ) + + now = datetime(2026, 7, 15, 3, 0, 0, tzinfo=timezone.utc) + # 1 packet at 18:00 yesterday UTC — older than the 6h window. + _make_reception( + db_session, + None, + "ancient", + ["AA", "BB"], + received_at=now - timedelta(hours=9), + ) + db_session.commit() + + results = evaluate_route_history( + db_session, route, days=3, include_today=True, now=now + ) + + today_entry = results[-1] + assert today_entry[0] == now.date() + assert today_entry[3] == 0 + assert today_entry[1] == RouteQuality.UNKNOWN.value + # --------------------------------------------------------------------------- # Branch-coverage tests (guards, exception handlers, optional-param paths)