diff --git a/docs/plans/20260703-1330-dashboard-packets-widget/plan.md b/docs/plans/20260703-1330-dashboard-packets-widget/plan.md new file mode 100644 index 0000000..5a7ae83 --- /dev/null +++ b/docs/plans/20260703-1330-dashboard-packets-widget/plan.md @@ -0,0 +1,293 @@ +# Dashboard & Homepage Packets Widget + +## Summary + +Surface raw-packet volume in the two overview surfaces of the web app. On the +**homepage**, add a fourth stat widget ("Packets") to the existing stats panel, +gated on the `packets` feature flag, and reduce the vertical height of all +homepage stat cards. On the **dashboard** page, remove the top row of summary +stat boxes entirely and instead fold each entity's headline number into the +top-right corner of its own chart card (same font size as the old stat value), +then add a fourth Packets chart so a `packets`-enabled deployment renders a +four-column chart grid. + +The raw-packets subsystem already exists end-to-end (model, collector +ingestion, `/packets` + `/packet-groups` API, feature flag, CSS color, icon, +i18n label). The only missing pieces are packet counts in the +`DashboardStats` payload, a daily packet-activity endpoint, and the frontend +wiring. + +## Background & Motivation + +The Raw Packets feature (`docs/plans/20260612-2014-raw-packets-feature`) shipped +a `raw_packets` table that captures every inbound MeshCore packet as it arrives +over the LetsMesh `packets` feed, plus a dedicated `/packets` browse page. That +data is fully available to operators but is **invisible on the two overview +pages**: the homepage stats panel shows Nodes / Advertisements / Messages, and +the dashboard page shows the same three entities as both stat boxes and charts. +Packet volume — often the highest-volume signal on a busy mesh — has no +presence on either overview. + +Concurrently, the dashboard page duplicates information: each entity appears +once as a big number in a stat box and again as a chart immediately below it. +Consolidating the number into the chart card header removes the duplication, +reclaims vertical space, and produces a cleaner, chart-first overview that +naturally extends to a fourth (Packets) column when the feature is enabled. + +Recent git history shows sustained UI polish work (panel-accent redesign, mobile +nav, typography adoption in `479c263`, `510612d`, `cb677b3`), so this fits the +current direction of tightening the overview surfaces. + +## Goals + +- Add a Packets stat widget to the homepage stats panel, gated on + `features.packets !== false`, showing the last-7-days packet count. +- Reduce the vertical footprint of homepage stat cards. +- Remove the dashboard's top stat-box row and render each entity's headline + number in the top-right corner of its chart card (matching the old stat-value + font size). +- Add a Packets chart to the dashboard, producing a 4-column chart grid when + `packets` is enabled (3 columns otherwise). +- Expose packet counts and daily packet activity from the backend without new + models, migrations, or config flags. + +## Non-Goals + +- No changes to the `/packets` browse page, filters, or detail view. +- No new feature flags, config variables, or i18n keys (all required strings and + the `feature_packets` flag already exist). +- No changes to channel-visibility / role-based redaction for packet payloads + (the new count/activity endpoints are role-independent volume metrics). +- No alteration of the homepage combined activity chart (adverts + messages). +- No database migrations (reuses the existing `raw_packets` table). + +## Requirements + +### Functional Requirements + +- **FR-1** — When `features.packets !== false`, the homepage stats panel renders + a fourth widget titled "Packets" (iconPackets, green accent), showing + `stats.packets_7d` with the description "last 7 days", ordered after Messages. +- **FR-2** — When `features.packets === false`, no Packets widget appears on the + homepage and no packets data is fetched beyond the single `/stats` call. +- **FR-3** — Homepage stat cards are visibly shorter than today (reduced vertical + padding and a smaller value font), affecting both the stats panel and the + members panel equally (global `renderStatCard` change). +- **FR-4** — The dashboard page no longer renders the top row of summary stat + boxes (Nodes / Advertisements / Messages). +- **FR-5** — Each dashboard chart card displays its headline number in the + top-right corner of the card, at the same font size as the former + `.stat-value` (`text-4xl`), colored with the entity accent: + - Nodes → `stats.total_nodes` + - Advertisements → `stats.advertisements_7d` + - Messages → `stats.messages_7d` + - Packets → `stats.packets_7d` +- **FR-6** — When `features.packets !== false`, the dashboard renders a fourth + chart card (Packets) with a 7-day daily line chart, producing a 4-column grid + at the `md` breakpoint and above. +- **FR-7** — `GET /api/v1/dashboard/stats` returns `total_packets` and + `packets_7d` integers. +- **FR-8** — `GET /api/v1/dashboard/packet-activity?days=N` returns a + `DailyActivity` payload (reused schema) of per-day raw-packet counts, capped + at 90 days, excluding today, with zero-filled missing days — identical + semantics to the existing `/activity` advertisement endpoint. + +### Technical Requirements + +- **TR-1** — Packet counts in `get_stats` use a single conditional-aggregation + query mirroring the advertisement block (`func.count()` + + `func.sum(case((received_at >= seven_days_ago, 1), else_=0))`), not two + separate queries. +- **TR-2** — The `/packet-activity` endpoint mirrors `/activity` exactly: + `func.date()` bucketing, `_date_bucket_key()` normalization for SQLite/Postgres + parity (see plan `20260616-2023-fix-postgres-charts-flatline`), default + `@cached` key (role-independent), `redis_cache_ttl_dashboard` TTL. +- **TR-3** — No role/channel-visibility filter on packet counts or activity + (raw packets are observer-level volume metrics; payload redaction stays on the + `/packets` list/detail endpoints only). +- **TR-4** — Frontend feature gating uses the established + `features.packets !== false` pattern (enabled by default). +- **TR-5** — `gridCols()` helper extended to return `md:grid-cols-4` for a count + of 4, so the chart grid scales from 1 → 2 → 3 → 4 columns. +- **TR-6** — `pageColors` gains a `packets` getter reading `--color-packets` + (which already exists in `app.css` for both light and dark themes). +- **TR-7** — `ChartColors` (charts.js) gains `packets` / `packetsFill` entries + so the Packets line chart uses the green accent. +- **TR-8** — Chart cleanup teardown in `dashboard.js` includes `'packetChart'`. +- **TR-9** — `renderStatCard` remains the single shared component; after this + change it is consumed only by `home.js` (dashboard drops its usage), so the + global shrink effectively only touches the homepage. + +## Implementation Plan + +### Phase 1: Backend — packet counts & activity endpoint + +- **`common/schemas/messages.py`** (`DashboardStats`, after `total_members` at + line 288): add two fields: + ```python + total_packets: int = Field(default=0, description="Total raw packets captured") + packets_7d: int = Field(default=0, description="Packets captured in last 7 days") + ``` + Reuse the existing `DailyActivity` schema for the activity endpoint (no new + schema). + +- **`api/routes/dashboard.py`**: + - Add `RawPacket` to the model imports (it is already exported from + `common/models`). + - In `get_stats`, after the advertisement count block (lines 137–148), add a + packet conditional-aggregation query and populate `total_packets` / + `packets_7d` in the `DashboardStats(...)` constructor (lines 268–282). + - Add a new route mirroring `/activity` (line 285): + ```python + @router.get("/packet-activity", response_model=DailyActivity) + @cached("dashboard/packet-activity", ttl_setting="redis_cache_ttl_dashboard") + def get_packet_activity(_: RequireRead, session: DbSession, request: Request, days: int = 30) -> DailyActivity: + ... + ``` + Body identical to `get_activity` but selecting from `RawPacket.received_at` + with no `_flood_only_filter` and no role/channel filter. + +### Phase 2: Frontend — shared component & color plumbing + +- **`spa/components.js`**: + - `pageColors` (line 183): add + `get packets() { return getComputedStyle(document.documentElement).getPropertyValue('--color-packets').trim(); }`. + - `renderStatCard` (line 803): global shrink — add `!py-2` to the root + `class` and change the value wrapper from `stat-value` to + `stat-value text-3xl` (drops the big number from ~2.5rem to ~1.875rem and + trims vertical padding). + +- **`charts.js`**: + - Add `packets` and `packetsFill` to `ChartColors` (read `--color-packets`). + - `initDashboardCharts` (line 203): accept a 4th `packetData` argument and, + when truthy, call `createLineChart('packetChart', packetData, + t('entities.packets'), ChartColors.packets, ChartColors.packetsFill, true)`. + +### Phase 3: Frontend — homepage widget + +- **`spa/pages/home.js`** (`renderStatsPanel`, after the Messages card at + line 164): append a fourth card gated on `features.packets !== false`: + ```js + ${features.packets !== false ? renderStatCard({ + icon: iconPackets('h-8 w-8'), + color: pageColors.packets, + title: t('entities.packets'), + value: stats.packets_7d, + description: t('time.last_7_days'), + }) : nothing} + ``` + `iconPackets` is already imported (line 7). The global `renderStatCard` shrink + applies automatically (no per-call change needed). + + `showStats` at line 228 must also include `|| features.packets !== false` so + the stats panel still renders when only packets is enabled (e.g. when a + deployment disables nodes, adverts, and messages but leaves packets on). + +### Phase 4: Frontend — dashboard restructure + +- **`spa/pages/dashboard.js`**: + - **Imports** (line 7–9): add `iconPackets` to the icons import: + ```js + import { iconNodes, iconAdvertisements, iconMessages, iconPackets, iconChannel } from '../icons.js'; + ``` + - `gridCols()` (line 105): add `if (count === 4) return 'md:grid-cols-4';`. + - **Delete** the stat-cards grid (lines 197–222) AND the `topCount`/`topGrid` + computation (lines 185–186), which become dead code. The chart grid render + gate (line 197) switches from `topCount > 0` to checking whether any feature + is enabled (the `visibleCount` computed inside `renderChartCards` already + drives the internal grid layout). + - `renderChartCards` (line 111): add `stats` and `showPackets` params. + Restructure each card's header into a flex row: + ```html +
${subtitle}
+...
` with a flex row: `${subtitle}
${t('common.no_entity_yet', { entity: t('entities.advertisements').toLowerCase() })}
`; + } + const rows = ads.map(ad => { + const friendlyName = ad.tag_name || ad.name; + const displayName = friendlyName || (ad.public_key.slice(0, 12) + '...'); + const keyLine = friendlyName + ? html`| ${t('entities.node')} | +${t('common.type')} | +${t('common.received')} | +${t('common.observers')} | +
|---|
${t('common.no_entity_yet', { entity: t('entities.advertisements').toLowerCase() })}
`; } - const rows = ads.slice(0, 5).map(ad => { + const rows = ads.map(ad => { const friendlyName = ad.tag_name || ad.name; const displayName = friendlyName || (ad.public_key.slice(0, 12) + '...'); const keyLine = friendlyName ? html`${t('time.over_time_last_7_days')}
+${t('time.over_time_last_7_days')}
+${t('time.per_day_last_7_days')}
+${t('time.per_day_last_7_days')}
+${t('time.per_day_last_7_days')}
+${t('time.per_day_last_7_days')}
+${t('time.per_day_last_7_days')}
+