Merge pull request #290 from ipnet-mesh/feat/dashboard-recent-adverts

feat(dashboard): enrich recent adverts card and add packets widget
This commit is contained in:
JingleManSweep
2026-07-03 14:34:24 +01:00
committed by GitHub
12 changed files with 1206 additions and 80 deletions
@@ -0,0 +1,293 @@
# Dashboard & Homepage Packets Widget
## Summary
Surface raw-packet volume in the two overview surfaces of the web app. On the
**homepage**, add a fourth stat widget ("Packets") to the existing stats panel,
gated on the `packets` feature flag, and reduce the vertical height of all
homepage stat cards. On the **dashboard** page, remove the top row of summary
stat boxes entirely and instead fold each entity's headline number into the
top-right corner of its own chart card (same font size as the old stat value),
then add a fourth Packets chart so a `packets`-enabled deployment renders a
four-column chart grid.
The raw-packets subsystem already exists end-to-end (model, collector
ingestion, `/packets` + `/packet-groups` API, feature flag, CSS color, icon,
i18n label). The only missing pieces are packet counts in the
`DashboardStats` payload, a daily packet-activity endpoint, and the frontend
wiring.
## Background & Motivation
The Raw Packets feature (`docs/plans/20260612-2014-raw-packets-feature`) shipped
a `raw_packets` table that captures every inbound MeshCore packet as it arrives
over the LetsMesh `packets` feed, plus a dedicated `/packets` browse page. That
data is fully available to operators but is **invisible on the two overview
pages**: the homepage stats panel shows Nodes / Advertisements / Messages, and
the dashboard page shows the same three entities as both stat boxes and charts.
Packet volume — often the highest-volume signal on a busy mesh — has no
presence on either overview.
Concurrently, the dashboard page duplicates information: each entity appears
once as a big number in a stat box and again as a chart immediately below it.
Consolidating the number into the chart card header removes the duplication,
reclaims vertical space, and produces a cleaner, chart-first overview that
naturally extends to a fourth (Packets) column when the feature is enabled.
Recent git history shows sustained UI polish work (panel-accent redesign, mobile
nav, typography adoption in `479c263`, `510612d`, `cb677b3`), so this fits the
current direction of tightening the overview surfaces.
## Goals
- Add a Packets stat widget to the homepage stats panel, gated on
`features.packets !== false`, showing the last-7-days packet count.
- Reduce the vertical footprint of homepage stat cards.
- Remove the dashboard's top stat-box row and render each entity's headline
number in the top-right corner of its chart card (matching the old stat-value
font size).
- Add a Packets chart to the dashboard, producing a 4-column chart grid when
`packets` is enabled (3 columns otherwise).
- Expose packet counts and daily packet activity from the backend without new
models, migrations, or config flags.
## Non-Goals
- No changes to the `/packets` browse page, filters, or detail view.
- No new feature flags, config variables, or i18n keys (all required strings and
the `feature_packets` flag already exist).
- No changes to channel-visibility / role-based redaction for packet payloads
(the new count/activity endpoints are role-independent volume metrics).
- No alteration of the homepage combined activity chart (adverts + messages).
- No database migrations (reuses the existing `raw_packets` table).
## Requirements
### Functional Requirements
- **FR-1** — When `features.packets !== false`, the homepage stats panel renders
a fourth widget titled "Packets" (iconPackets, green accent), showing
`stats.packets_7d` with the description "last 7 days", ordered after Messages.
- **FR-2** — When `features.packets === false`, no Packets widget appears on the
homepage and no packets data is fetched beyond the single `/stats` call.
- **FR-3** — Homepage stat cards are visibly shorter than today (reduced vertical
padding and a smaller value font), affecting both the stats panel and the
members panel equally (global `renderStatCard` change).
- **FR-4** — The dashboard page no longer renders the top row of summary stat
boxes (Nodes / Advertisements / Messages).
- **FR-5** — Each dashboard chart card displays its headline number in the
top-right corner of the card, at the same font size as the former
`.stat-value` (`text-4xl`), colored with the entity accent:
- Nodes → `stats.total_nodes`
- Advertisements → `stats.advertisements_7d`
- Messages → `stats.messages_7d`
- Packets → `stats.packets_7d`
- **FR-6** — When `features.packets !== false`, the dashboard renders a fourth
chart card (Packets) with a 7-day daily line chart, producing a 4-column grid
at the `md` breakpoint and above.
- **FR-7** — `GET /api/v1/dashboard/stats` returns `total_packets` and
`packets_7d` integers.
- **FR-8** — `GET /api/v1/dashboard/packet-activity?days=N` returns a
`DailyActivity` payload (reused schema) of per-day raw-packet counts, capped
at 90 days, excluding today, with zero-filled missing days — identical
semantics to the existing `/activity` advertisement endpoint.
### Technical Requirements
- **TR-1** — Packet counts in `get_stats` use a single conditional-aggregation
query mirroring the advertisement block (`func.count()` +
`func.sum(case((received_at >= seven_days_ago, 1), else_=0))`), not two
separate queries.
- **TR-2** — The `/packet-activity` endpoint mirrors `/activity` exactly:
`func.date()` bucketing, `_date_bucket_key()` normalization for SQLite/Postgres
parity (see plan `20260616-2023-fix-postgres-charts-flatline`), default
`@cached` key (role-independent), `redis_cache_ttl_dashboard` TTL.
- **TR-3** — No role/channel-visibility filter on packet counts or activity
(raw packets are observer-level volume metrics; payload redaction stays on the
`/packets` list/detail endpoints only).
- **TR-4** — Frontend feature gating uses the established
`features.packets !== false` pattern (enabled by default).
- **TR-5** — `gridCols()` helper extended to return `md:grid-cols-4` for a count
of 4, so the chart grid scales from 1 → 2 → 3 → 4 columns.
- **TR-6** — `pageColors` gains a `packets` getter reading `--color-packets`
(which already exists in `app.css` for both light and dark themes).
- **TR-7** — `ChartColors` (charts.js) gains `packets` / `packetsFill` entries
so the Packets line chart uses the green accent.
- **TR-8** — Chart cleanup teardown in `dashboard.js` includes `'packetChart'`.
- **TR-9** — `renderStatCard` remains the single shared component; after this
change it is consumed only by `home.js` (dashboard drops its usage), so the
global shrink effectively only touches the homepage.
## Implementation Plan
### Phase 1: Backend — packet counts & activity endpoint
- **`common/schemas/messages.py`** (`DashboardStats`, after `total_members` at
line 288): add two fields:
```python
total_packets: int = Field(default=0, description="Total raw packets captured")
packets_7d: int = Field(default=0, description="Packets captured in last 7 days")
```
Reuse the existing `DailyActivity` schema for the activity endpoint (no new
schema).
- **`api/routes/dashboard.py`**:
- Add `RawPacket` to the model imports (it is already exported from
`common/models`).
- In `get_stats`, after the advertisement count block (lines 137148), add a
packet conditional-aggregation query and populate `total_packets` /
`packets_7d` in the `DashboardStats(...)` constructor (lines 268282).
- Add a new route mirroring `/activity` (line 285):
```python
@router.get("/packet-activity", response_model=DailyActivity)
@cached("dashboard/packet-activity", ttl_setting="redis_cache_ttl_dashboard")
def get_packet_activity(_: RequireRead, session: DbSession, request: Request, days: int = 30) -> DailyActivity:
...
```
Body identical to `get_activity` but selecting from `RawPacket.received_at`
with no `_flood_only_filter` and no role/channel filter.
### Phase 2: Frontend — shared component & color plumbing
- **`spa/components.js`**:
- `pageColors` (line 183): add
`get packets() { return getComputedStyle(document.documentElement).getPropertyValue('--color-packets').trim(); }`.
- `renderStatCard` (line 803): global shrink — add `!py-2` to the root
`class` and change the value wrapper from `stat-value` to
`stat-value text-3xl` (drops the big number from ~2.5rem to ~1.875rem and
trims vertical padding).
- **`charts.js`**:
- Add `packets` and `packetsFill` to `ChartColors` (read `--color-packets`).
- `initDashboardCharts` (line 203): accept a 4th `packetData` argument and,
when truthy, call `createLineChart('packetChart', packetData,
t('entities.packets'), ChartColors.packets, ChartColors.packetsFill, true)`.
### Phase 3: Frontend — homepage widget
- **`spa/pages/home.js`** (`renderStatsPanel`, after the Messages card at
line 164): append a fourth card gated on `features.packets !== false`:
```js
${features.packets !== false ? renderStatCard({
icon: iconPackets('h-8 w-8'),
color: pageColors.packets,
title: t('entities.packets'),
value: stats.packets_7d,
description: t('time.last_7_days'),
}) : nothing}
```
`iconPackets` is already imported (line 7). The global `renderStatCard` shrink
applies automatically (no per-call change needed).
`showStats` at line 228 must also include `|| features.packets !== false` so
the stats panel still renders when only packets is enabled (e.g. when a
deployment disables nodes, adverts, and messages but leaves packets on).
### Phase 4: Frontend — dashboard restructure
- **`spa/pages/dashboard.js`**:
- **Imports** (line 79): add `iconPackets` to the icons import:
```js
import { iconNodes, iconAdvertisements, iconMessages, iconPackets, iconChannel } from '../icons.js';
```
- `gridCols()` (line 105): add `if (count === 4) return 'md:grid-cols-4';`.
- **Delete** the stat-cards grid (lines 197222) AND the `topCount`/`topGrid`
computation (lines 185186), which become dead code. The chart grid render
gate (line 197) switches from `topCount > 0` to checking whether any feature
is enabled (the `visibleCount` computed inside `renderChartCards` already
drives the internal grid layout).
- `renderChartCards` (line 111): add `stats` and `showPackets` params.
Restructure each card's header into a flex row:
```html
<div class="flex items-start justify-between gap-2">
<div>
<h2 class="card-title text-base"> ${icon} ${title} </h2>
<p class="text-xs opacity-80"> ${subtitle} </p>
</div>
<div class="text-4xl font-bold leading-none" style="color: var(--color-<entity>)">
${value}
</div>
</div>
```
— **Nodes**: title `t('entities.nodes')`, subtitle `t('time.over_time_last_7_days')`, value `stats.total_nodes`, color `--color-nodes`.
— **Advertisements**: title `t('entities.advertisements')`, subtitle `t('time.per_day_last_7_days')`, value `stats.advertisements_7d`, color `--color-adverts`.
— **Messages**: title `t('entities.messages')`, subtitle `t('time.per_day_last_7_days')`, value `stats.messages_7d`, color `--color-messages`.
Add a Packets card (`#packetChart`, color `--color-packets`, `iconPackets`,
title `t('entities.packets')`, subtitle `t('time.per_day_last_7_days')`,
value `stats.packets_7d`). Include packets in `visibleCount` for the
grid-cols calculation.
- Main `render` (line 160): add `const showPackets = features.packets !== false;`.
Add `/api/v1/dashboard/packet-activity?days=7` to the `Promise.all` (line
170). Pass `stats` and `showPackets` to `renderChartCards`. Pass
`showPackets ? packetActivity : null` as the 4th arg to
`window.initDashboardCharts`. Add `'packetChart'` to the `chartIds` cleanup
array (line 248). Remove `renderStatCard` from the imports (line 5) once the
stat grid is gone.
### Phase 5: Tests & verification
- **`tests/test_api/test_dashboard.py`**:
- Assert `packets_7d` (and `total_packets`) are present in the `/stats`
response.
- Add a test for `/dashboard/packet-activity`: create an observer `Node` +
`RawPacket` rows across two days, request `?days=7`, assert correct daily
counts and zero-fill. Mirror the existing advertisement-activity test's
fixture style.
- Run `pytest --no-cov tests/test_api/test_dashboard.py tests/test_web/`.
- Run `pre-commit run --all-files`.
- `make build` then visually verify `/` (4 shorter stat cards when packets
enabled; 3 when disabled) and `/dashboard` (no stat boxes; numbers in chart
corners; 4-column chart grid when packets enabled, 3 when disabled).
## Open Questions
- None remaining. Count window (last 7 days), stat-card height approach (global
shrink), and dashboard consolidation are all confirmed decisions.
## References
- `docs/plans/20260612-2014-raw-packets-feature/plan.md` — the Raw Packets
feature this plan builds on (model, collector, `/packets` API, feature flag).
- `docs/plans/20260616-2023-fix-postgres-charts-flatline/plan.md` — the
`_date_bucket_key()` dialect-neutral date handling that the new
`/packet-activity` endpoint must reuse.
- `docs/plans/20260506-1330-members-widget/plan.md` — prior homepage stats-panel
work (operator/member widgets via `renderStatCard`).
- `docs/plans/20260609-2106-redis-api-cache/plan.md` — the `@cached` decorator
and `redis_cache_ttl_dashboard` TTL used by the new endpoint.
## Review
**Status**: Approved with Changes
**Reviewed**: 2026-07-03
### Resolutions
- **`showStats` missing `features.packets`**: Confirmed absent at
`home.js:228`. Added explicit instruction to Phase 3 to include
`|| features.packets !== false`.
- **`iconPackets` missing from dashboard.js imports**: Confirmed absent from
`dashboard.js:7-9`. Added import instruction to Phase 4.
- **`topCount`/`topGrid` dead code after stat-cards removal**: Confirmed
variables at `dashboard.js:185-186` would become unreferenced. Added
explicit removal to Phase 4 and clarified the chart grid render gate should
use the internal `visibleCount` instead.
- **Line reference approximations fixed**: `DashboardStats` field insertion
point corrected from ~287 to after `total_members` (line 288). Advert count
block corrected from ~148 to 137148.
- **`received_at` index exists**: Confirmed `ix_raw_packets_received_at` at
`raw_packet.py:104`. No migration needed for `/packet-activity` query
performance. `TR-1` query with `received_at >= seven_days_ago` will use
this index.
- **All i18n keys exist**: Confirmed `time.last_7_days` and
`time.per_day_last_7_days` in both `en.json` (lines 134135) and `nl.json`
(lines 124125). No new i18n keys needed.
- **Chart subtitle for Nodes**: Verified currently uses
`time.over_time_last_7_days` (different from adverts/messages). Added
per-card subtitle specifications to Phase 4 to prevent accidental
homogenization.
### Remaining Action Items
- None. All review findings resolved in-plan.
@@ -0,0 +1,80 @@
# Tasks: Dashboard & Homepage Packets Widget
> Generated from `plan.md` on 2026-07-03
## Phase 1: Backend — Packet Counts & Activity Endpoint
- [x] Add `total_packets` and `packets_7d` fields to `DashboardStats` schema
- [x] In `meshcore_hub/common/schemas/messages.py`, after `total_members` (line 288), add `total_packets: int = Field(default=0)` and `packets_7d: int = Field(default=0)`
- [x] Import `RawPacket` model in dashboard routes
- [x] In `meshcore_hub/api/routes/dashboard.py`, add `from meshcore_hub.common.models import RawPacket` alongside existing model imports
- [x] Add packet conditional-aggregation query to `get_stats`
- [x] After the advertisement count block (lines 137148), add a query mirroring that pattern: `func.count(RawPacket.id).label("total_packets")` + `func.sum(case((RawPacket.received_at >= seven_days_ago, 1), else_=0)).label("packets_7d")`
- [x] Populate `total_packets` and `packets_7d` in the `DashboardStats(...)` constructor (lines 268282)
- [x] Add `/packet-activity` daily activity endpoint
- [x] Mirror `get_activity` (line 285): select `func.date(RawPacket.received_at)` buckets from `RawPacket`, apply `start_date`/`end_date` range excluding today, use `_date_bucket_key()` for SQLite/Postgres parity, zero-fill missing days using the existing loop pattern
- [x] Return `DailyActivity` response model (reused schema)
- [x] Decorate with `@cached("dashboard/packet-activity", ttl_setting="redis_cache_ttl_dashboard")` (no role/channel filter; default cache key like `/activity`)
## Phase 2: Frontend — Shared Component & Color Plumbing
- [x] Add `packets` getter to `pageColors`
- [x] In `meshcore_hub/web/static/js/spa/components.js`, after the `messages` getter (line 190), add `get packets() { return getComputedStyle(document.documentElement).getPropertyValue('--color-packets').trim(); }`
- [x] Shrink `renderStatCard` globally
- [x] In `meshcore_hub/web/static/js/spa/components.js`, in `renderStatCard` (line 803), add `!py-2` to the root element's `class` attribute
- [x] Change `<div class="stat-value">` to `<div class="stat-value text-3xl">`
- [x] Add `packets` / `packetsFill` to `ChartColors`
- [x] In `meshcore_hub/web/static/js/charts.js`, after `messagesFill`, add `packets: getCSSColor('--color-packets', ...)` and `packetsFill: getCSSColor('--color-packets', ...)` (same color for line and fill, matching the green accent)
- [x] Extend `initDashboardCharts` for packet data
- [x] Add a 4th `packetData` parameter to `initDashboardCharts` (line 203)
- [x] When `packetData` is truthy, call `createLineChart('packetChart', packetData, t('entities.packets'), ChartColors.packets, ChartColors.packetsFill, true)` after the existing chart calls
## Phase 3: Frontend — Homepage Widget
- [x] Add Packets stat card to `renderStatsPanel`
- [x] In `meshcore_hub/web/static/js/spa/pages/home.js`, after the Messages card (line 164), append a fourth card gated on `features.packets !== false` showing `stats.packets_7d` with `iconPackets`, `pageColors.packets`, `t('entities.packets')`, and `t('time.last_7_days')`
- [x] Update `showStats` gate to include packets
- [x] In `home.js` at line 228, append `|| features.packets !== false` to the `showStats` expression
## Phase 4: Frontend — Dashboard Restructure
- [x] Update imports and extend `gridCols`
- [x] In `meshcore_hub/web/static/js/spa/pages/dashboard.js`, add `iconPackets` to the `icons.js` import (line 8)
- [x] Remove `renderStatCard` from the `components.js` import (line 5) since the dashboard no longer uses it
- [x] In `gridCols()` (line 105), add `if (count === 4) return 'md:grid-cols-4';`
- [x] Remove stat-cards grid and dead variables
- [x] Delete the `topCount` / `topGrid` computation (lines 185186)
- [x] Delete the stat-cards grid template block (lines 197222)
- [x] Update the render gate (line 197) from `topCount > 0` to `(showNodes || showAdverts || showMessages || showPackets)`
- [x] Restructure chart card headers with corner numbers
- [x] In `renderChartCards`, add `stats` and `showPackets` parameters
- [x] For each existing card (Nodes, Advertisements, Messages), replace `<h2 class="card-title text-base">...</h2><p class="text-xs opacity-80">...</p>` with a flex row: `<div class="flex items-start justify-between gap-2"><div><h2 class="card-title text-base">${icon} ${title}</h2><p class="text-xs opacity-80">${subtitle}</p></div><div class="text-4xl font-bold leading-none" style="color: var(...)">${value}</div></div>`
- [x] Nodes: title `t('entities.nodes')`, subtitle `t('time.over_time_last_7_days')`, value `stats.total_nodes`, color `--color-nodes`
- [x] Advertisements: title `t('entities.advertisements')`, subtitle `t('time.per_day_last_7_days')`, value `stats.advertisements_7d`, color `--color-adverts`
- [x] Messages: title `t('entities.messages')`, subtitle `t('time.per_day_last_7_days')`, value `stats.messages_7d`, color `--color-messages`
- [x] Add Packets chart card to `renderChartCards`
- [x] Append a Packets card (gated on `showPackets`) with `#packetChart`, `--color-packets`, `iconPackets`, title `t('entities.packets')`, subtitle `t('time.per_day_last_7_days')`, value `stats.packets_7d`
- [x] Include `showPackets` in the `visibleCount` computation
- [x] Wire up packets data fetching, chart init, and cleanup
- [x] In `render` (line 160), add `const showPackets = features.packets !== false;`
- [x] Add `/api/v1/dashboard/packet-activity?days=7` to the `Promise.all` call (line 170)
- [x] Destructure `packetActivity` from the response, pass `stats` and `showPackets` to `renderChartCards`
- [x] Pass `showPackets ? packetActivity : null` as the 4th argument to `window.initDashboardCharts`
- [x] Add `'packetChart'` to the `chartIds` cleanup array (line 248)
## Phase 5: Tests & Verification
- [x] Add backend tests for packets in `/stats` response
- [x] In `tests/test_api/test_dashboard.py`, add a test asserting `total_packets` and `packets_7d` are present in the `/stats` response and have the correct defaults (0 when no packets exist)
- [x] Add backend test for `/packet-activity` endpoint
- [x] Create a test fixture with an observer `Node` and `RawPacket` rows across two days
- [x] Request `/packet-activity?days=7`, assert correct per-day counts and zero-fill for days without data
- [x] Mirror the existing `/activity` test fixture style
- [x] Run targeted backend and web tests
- [x] `pytest --no-cov tests/test_api/test_dashboard.py tests/test_web/` — expect all passing
- [x] Run full lint pass
- [x] `pre-commit run --all-files` — expect clean
- [x] Build and visually verify
- [x] Run `make build` to rebuild the frontend container
- [x] Verify homepage: 4 shorter stat cards when packets enabled; 3 when disabled
- [x] Verify dashboard: no stat boxes; numbers in chart card corners; 4-column chart grid when packets enabled, 3 when disabled
@@ -0,0 +1,322 @@
# Dashboard Recent Adverts Card Improvements
## Summary
The Recent Adverts card on the dashboard page currently shows a compact table of
5 advertisements with three columns: Node (name + truncated key), Type (a
satellite-dish emoji derived from `adv_type`), and Time. This plan enriches the
card to better match the full Advertisement list page by replacing the emoji
column with a route-type text badge (e.g. "Flood", "Relay"), adding an Observers
column (observer count badges with name tooltips), and increasing the displayed
row count from 5 to 10. The Time column is retained.
This requires a small backend change to include `route_type` and `observers` in
the `RecentAdvertisement` schema and the `get_stats` query, plus a frontend
rewrite of the `renderRecentAds` function to consume the new fields. A local
`routeTypeBadge` helper from `advertisements.js` is promoted to a shared export
in `components.js` to avoid duplication.
## Background & Motivation
The dashboard was recently restructured (plan
`20260703-1330-dashboard-packets-widget`) to remove the top stat-box row and
fold headline numbers into chart card corners. The bottom section retains two
cards: Recent Adverts and Recent Channel Messages. The Recent Adverts card is
the primary "what just happened" surface on the dashboard, but its current
information density is low — it shows only a type emoji and a time, omitting two
of the most useful columns from the full Advertisement list page: the route type
badge (which distinguishes flood vs. direct/relay) and the observer count (how
many stations heard the advert).
The recent git history shows sustained UI polish work (panel-accent redesign,
mobile nav clickthrough fix, typography adoption in commits `479c263`,
`510612d`, `cb677b3`), so this fits the current direction of tightening the
overview surfaces to be more informative at a glance.
The observer infrastructure already exists: `fetch_observers_for_events()` in
`observer_utils.py` batch-fetches `ObserverInfo` objects by `event_hash` and is
already used by the advertisements, messages, telemetry, and trace-path list
endpoints. The Advertisement model has `route_type` and `event_hash` columns.
The frontend `observerIcons()` helper in `components.js` already renders the
count-badge-with-tooltip. The only missing piece is wiring these into the
dashboard's recent-adverts data path and rendering.
## Goals
- Replace the satellite-dish emoji column in the dashboard Recent Adverts card
with a route-type text badge (Flood / Relay / Zero-hop / Direct relay),
matching the Advertisement list page's badge rendering.
- Add an Observers column showing observer count badges (with name tooltips),
matching the Advertisement list page's observer rendering.
- Increase the displayed recent adverts from 5 to 10 (the backend already
fetches 10; the frontend currently slices to 5).
- Retain the Time column (confirmed by the user).
- Avoid code duplication by sharing the `routeTypeBadge` helper between the
dashboard and the advertisements list page.
## Non-Goals
- No changes to the Advertisement list page rendering or behavior (it already
has route type badges and observer columns). It only changes its import source
for `routeTypeBadge`.
- No changes to the homepage stat cards or dashboard chart cards (those were
addressed in the prior packets-widget plan).
- No new API endpoints, feature flags, config variables, i18n keys, or database
migrations.
- No changes to the Recent Channel Messages card.
- No pagination, sorting, filtering, or observer-filter toggle on the dashboard
card (the dashboard is a read-only overview; full filtering lives on the
Advertisement list page).
## Requirements
### Functional Requirements
- **FR-1** — The dashboard Recent Adverts card displays up to 10 advertisement
rows (currently 5). The backend `/stats` endpoint already returns 10; the
frontend `ads.slice(0, 5)` call must be removed.
- **FR-2** — Each row has four columns: **Node** (display name + truncated
public key underneath, linked to `/nodes/{public_key}`), **Type** (route-type
badge), **Time** (time-only formatted, as today), **Observers** (observer
count badge or dash).
- **FR-3** — The Type column renders a route-type badge using the same logic as
the Advertisement list page: `flood` → blue "Flood" badge;
`transport_flood` → blue "Relay" badge; `direct` → green "Zero-hop" badge;
`transport_direct` → green "Direct relay" badge; `null`/unknown → empty
(nothing rendered).
- **FR-4** — The Observers column renders an observer count badge
(`observerIcons()`) when the advert has `observers.length >= 1`, a faded
satellite emoji when only the legacy `observed_by` field is present (no
event-observer rows), and a faded dash when neither is present — mirroring the
Advertisement list page's three-way fallback.
- **FR-5** — The `GET /api/v1/dashboard/stats` response includes `route_type`,
`observers`, and `observed_by` on each item in `recent_advertisements`.
### Technical Requirements
- **TR-1** — The `RecentAdvertisement` schema gains three fields:
`route_type: Optional[str]`, `observers: list[ObserverInfo]` (default
empty list), and `observed_by: Optional[str]`. The `ObserverInfo` schema is
reused as-is (already defined in the same module).
- **TR-2** — The `get_stats` dashboard route collects `event_hash` values from
the fetched recent adverts and calls `fetch_observers_for_events(session,
"advertisement", event_hashes)` to batch-resolve observer data in a single
query — the same pattern used by the advertisements list endpoint.
- **TR-3** — The `routeTypeBadge` function is moved from a local definition in
`advertisements.js` to an exported function in `components.js`, so both the
dashboard and the advertisements page share a single implementation.
`advertisements.js` updates its import accordingly.
- **TR-4** — The frontend `renderRecentAds` function imports `observerIcons`
and `routeTypeBadge` from `components.js` and uses them in the new column
layout. The `typeEmoji` import (used only for the old emoji column) is removed
from `dashboard.js` if it becomes unused after the change.
- **TR-5** — Observer fallback logic matches the Advertisement list page: if
`ad.observers` is a non-empty array, render `observerIcons(ad.observers)`;
otherwise if `ad.observed_by` is truthy, render a faded satellite emoji; else
render a faded dash.
- **TR-6** — No new i18n keys needed. The route-type badge text ("Flood",
"Relay", "Zero-hop", "Direct relay") is hardcoded in the existing
`routeTypeBadge` function and is not i18n-ized today. The column headers reuse
existing keys: `entities.node`, `common.type`, `common.received`,
`common.observers`.
## Implementation Plan
### Phase 1: Backend — schema & data
- **`src/meshcore_hub/common/schemas/messages.py`** — `RecentAdvertisement`
(line 237): add three fields after `received_at`:
```python
route_type: Optional[str] = Field(default=None, description="Route type")
observers: list[ObserverInfo] = Field(
default_factory=list, description="All observers that captured this advertisement"
)
observed_by: Optional[str] = Field(
default=None, description="Observing interface node public key"
)
```
`ObserverInfo` is already defined earlier in the same module (line 9), so no
new import needed.
- **`src/meshcore_hub/api/routes/dashboard.py`** — `get_stats()`:
- Add `fetch_observers_for_events` to the import from `observer_utils` (line
17 already imports `resolve_sender_names` from that module).
- After the recent-adverts query (line 164174) and before building the
`RecentAdvertisement` list (line 202), collect event hashes and fetch
observers:
```python
ad_event_hashes = [ad.event_hash for ad in recent_ads if ad.event_hash]
observers_by_hash = fetch_observers_for_events(
session, "advertisement", ad_event_hashes
)
```
- Resolve `observer_node_id` → `public_key` for the `observed_by` fallback.
Collect observer node IDs from the ads and look up their public keys:
```python
observer_node_ids = [ad.observer_node_id for ad in recent_ads if ad.observer_node_id]
observer_pk_map: dict[str, str] = {}
if observer_node_ids:
obs_query = select(Node.id, Node.public_key).where(
Node.id.in_(observer_node_ids)
)
for node_id, public_key in session.execute(obs_query).all():
observer_pk_map[node_id] = public_key
```
- In the `RecentAdvertisement(...)` constructor (line 203210), add:
```python
route_type=ad.route_type,
observers=observers_by_hash.get(ad.event_hash, []) if ad.event_hash else [],
observed_by=observer_pk_map.get(ad.observer_node_id) if ad.observer_node_id else None,
```
### Phase 2: Frontend — shared component extraction
- **`src/meshcore_hub/web/static/js/spa/components.js`** — Add an exported
`routeTypeBadge(routeType)` function with the same logic currently in
`advertisements.js:12-23`:
```js
export function routeTypeBadge(routeType) {
if (!routeType) return nothing;
if (routeType === 'flood' || routeType === 'transport_flood') {
return html`<span class="badge badge-sm badge-info">${routeType === 'flood' ? 'Flood' : 'Relay'}</span>`;
}
if (routeType === 'direct' || routeType === 'transport_direct') {
return html`<span class="badge badge-sm badge-success">${routeType === 'direct' ? 'Zero-hop' : 'Direct relay'}</span>`;
}
return nothing;
}
```
Place it near the other badge/icon helpers (e.g. after `observerIcons` at line
559). Ensure `html` and `nothing` are already imported (they are).
- **`src/meshcore_hub/web/static/js/spa/pages/advertisements.js`** — Remove the
local `routeTypeBadge` function (lines 1223) and add it to the import from
`../components.js` (line 29).
### Phase 3: Frontend — dashboard card rewrite
- **`src/meshcore_hub/web/static/js/spa/pages/dashboard.js`** —
`renderRecentAds()` (lines 3468):
- **Imports** (lines 26): add `observerIcons` and `routeTypeBadge` to the
destructured import from `../components.js`. Remove `typeEmoji` from the
import if it is no longer used elsewhere in the file (verify: `typeEmoji`
is only used at line 51 in `renderRecentAds`, so it can be dropped).
- **Remove the `.slice(0, 5)`** call (line 38) so all 10 rows render.
- **Rewrite the table** to four columns. Replace the current `<thead>` and
row template:
```js
function renderRecentAds(ads) {
if (!ads || ads.length === 0) {
return html`<p class="text-sm opacity-70">${t('common.no_entity_yet', { entity: t('entities.advertisements').toLowerCase() })}</p>`;
}
const rows = ads.map(ad => {
const friendlyName = ad.tag_name || ad.name;
const displayName = friendlyName || (ad.public_key.slice(0, 12) + '...');
const keyLine = friendlyName
? html`<div class="text-xs opacity-50 font-mono">${ad.public_key.slice(0, 12)}...</div>`
: nothing;
let observersBlock;
if (ad.observers && ad.observers.length >= 1) {
observersBlock = html`${observerIcons(ad.observers)}`;
} else if (ad.observed_by) {
observersBlock = html`<span class="opacity-50">\u{1F4E1}</span>`;
} else {
observersBlock = html`<span class="opacity-50">-</span>`;
}
return html`<tr>
<td>
<a href="/nodes/${ad.public_key}" class="link link-hover">
<div class="font-medium">${displayName}</div>
</a>
${keyLine}
</td>
<td>${routeTypeBadge(ad.route_type)}</td>
<td class="text-right text-sm opacity-70">${formatTimeOnly(ad.received_at)}</td>
<td>${observersBlock}</td>
</tr>`;
});
return html`<div class="overflow-x-auto">
<table class="table table-sm w-full">
<thead>
<tr>
<th>${t('entities.node')}</th>
<th>${t('common.type')}</th>
<th class="text-right">${t('common.received')}</th>
<th>${t('common.observers')}</th>
</tr>
</thead>
<tbody>${rows}</tbody>
</table>
</div>`;
}
```
### Phase 4: Tests & verification
- **`tests/test_api/test_dashboard.py`**:
- **Existing test `test_recent_ads_excludes_direct`** (line 659): the flood ad
already has `route_type="flood"`. Add assertions:
```python
assert data["recent_advertisements"][0]["route_type"] == "flood"
assert data["recent_advertisements"][0]["observers"] == []
assert data["recent_advertisements"][0]["observed_by"] is None
```
- **Existing test `test_recent_advertisements_includes_tag_name`** (line 847):
the ad has `route_type="flood"` but no `event_hash`. Assert:
```python
assert data["recent_advertisements"][0]["route_type"] == "flood"
assert data["recent_advertisements"][0]["observers"] == []
assert data["recent_advertisements"][0]["observed_by"] is None
```
- **New test `test_recent_advertisements_includes_observers`**: Create an
observer `Node`, an `Advertisement` with an `event_hash`, and an
`EventObserver` row linking them. Assert that `recent_advertisements[0]` has
`observers` with length 1 and the observer's `public_key` matches. Mirror
the fixture style of `test_api/test_advertisements.py:40-66`.
- **New test `test_recent_advertisements_includes_observed_by`**: Create an
observer `Node` (public key `OBSERVERKEY...`), set
`advertisement.observer_node_id = observer_node.id`, no `event_hash` / no
`EventObserver` row. Assert that `data["recent_advertisements"][0]` has
`observers == []` and `observed_by == "OBSERVERKEY..."`, covering the
satellite-emoji fallback path.
- Run `pytest --no-cov tests/test_api/test_dashboard.py`.
- Run `pre-commit run --all-files`.
- `make build` then visually verify the `/dashboard` Recent Adverts card: route
type badges appear, observer count badges appear, 10 rows shown, time column
retained.
## Review
**Status**: Approved
**Reviewed**: 2026-07-03
### Resolutions
- **`observed_by` on `RecentAdvertisement`**: Add the field. The backend resolves
`advertisement.observer_node_id → Node.public_key` for each recent ad (batched
query) and populates `observed_by` on the schema. The frontend uses the
full three-way fallback: `ad.observers` array → `ad.observed_by` (satellite
emoji) → dash. A new test `test_recent_advertisements_includes_observed_by`
covers this path.
### Remaining Action Items
- (none)
## References
- `docs/plans/20260703-1330-dashboard-packets-widget/plan.md` — prior dashboard
restructure (removed stat boxes, added chart-card corner numbers, added
Packets chart). This plan continues polishing the dashboard's bottom section.
- `docs/plans/20260614-1220-observer-filter-badges/plan.md` — introduced the
`observerIcons` helper and observer badge rendering pattern used here.
- `docs/plans/20260515-1900-recent-adverts-timestamps/plan.md` — prior work on
the recent-adverts data path (added `public_key` filter to the advertisements
API for the node-detail page; the dashboard path is separate).
- `src/meshcore_hub/api/observer_utils.py` — `fetch_observers_for_events()`,
the batch observer-resolution utility reused by this plan.
- `src/meshcore_hub/api/routes/advertisements.py:210-243` — the reference
implementation of observer/route-type population in the advertisements list
endpoint.
@@ -0,0 +1,61 @@
# Tasks: Dashboard Recent Adverts Card Improvements
> Generated from `plan.md` on 2026-07-03
## Backend: Schema & Data
- [x] Add `route_type`, `observers`, and `observed_by` fields to `RecentAdvertisement` schema
- [x] Open `src/meshcore_hub/common/schemas/messages.py`
- [x] Add `route_type: Optional[str] = Field(default=None, description="Route type")` after `received_at` (line 244)
- [x] Add `observers: list[ObserverInfo] = Field(default_factory=list, description="All observers that captured this advertisement")` after `route_type`
- [x] Add `observed_by: Optional[str] = Field(default=None, description="Observing interface node public key")` after `observers`
- [x] Wire observer data into `get_stats` dashboard endpoint
- [x] Open `src/meshcore_hub/api/routes/dashboard.py`
- [x] Add `fetch_observers_for_events` to the import from `observer_utils` (line 17)
- [x] After the recent-ads query (line 174) and before building `RecentAdvertisement` list (line 202), collect event hashes and call `fetch_observers_for_events(session, "advertisement", ad_event_hashes)`
- [x] Collect `observer_node_id` values from recent ads and batch-query `Node.id, Node.public_key` to build an `observer_pk_map` dict
- [x] Pass `route_type=ad.route_type`, `observers=...`, and `observed_by=...` into the `RecentAdvertisement(...)` constructor (line 203)
## Frontend: Shared Component Extraction
- [x] Extract `routeTypeBadge` from `advertisements.js` into `components.js`
- [x] Open `src/meshcore_hub/web/static/js/spa/components.js`
- [x] Add exported `routeTypeBadge(routeType)` function after `observerIcons` (near line 559) with the same flood/direct/transport logic
- [x] Open `src/meshcore_hub/web/static/js/spa/pages/advertisements.js`
- [x] Remove the local `routeTypeBadge` function (lines 1223)
- [x] Add `routeTypeBadge` to the destructured import from `../components.js` (line 29)
## Frontend: Dashboard Card Rewrite
- [x] Rewrite `renderRecentAds` in `dashboard.js`
- [x] Open `src/meshcore_hub/web/static/js/spa/pages/dashboard.js`
- [x] Add `observerIcons` and `routeTypeBadge` to the import from `../components.js` (line 26)
- [x] Remove `typeEmoji` from the import (only used by old emoji column; verify no other callers)
- [x] Remove `.slice(0, 5)` call on line 38 so all 10 rows render
- [x] Rewrite the table to four columns: Node, Type, Time, Observers
- [x] Implement three-way observer fallback in the row template: `ad.observers` array → `ad.observed_by` satellite emoji → dash
- [x] Use `routeTypeBadge(ad.route_type)` for the Type column
- [x] Use `observerIcons(ad.observers)` for the Observers column when observers array is non-empty
- [x] Retain the existing `formatTimeOnly(ad.received_at)` for the Time column
## Tests
- [x] Update existing dashboard tests for new fields
- [x] Open `tests/test_api/test_dashboard.py`
- [x] In `test_recent_ads_excludes_direct` (line 659): add assertions for `route_type == "flood"`, `observers == []`, `observed_by is None`
- [x] In `test_recent_advertisements_includes_tag_name` (line 847): add assertions for `route_type == "flood"`, `observers == []`, `observed_by is None`
- [x] Add new test for observer population
- [x] Add `test_recent_advertisements_includes_observers`: create observer Node, Advertisement with `event_hash`, and EventObserver row; assert `observers` has length 1 and correct `public_key`
- [x] Add new test for `observed_by` fallback
- [x] Add `test_recent_advertisements_includes_observed_by`: create observer Node, Advertisement with `observer_node_id` set but no `event_hash` / EventObserver; assert `observers == []` and `observed_by` matches observer public key
## Verification
- [ ] Run dashboard API tests: `pytest --no-cov tests/test_api/test_dashboard.py -v`
- [ ] Run full quality checks: `pre-commit run --all-files`
- [ ] Rebuild stack: `make build`
- [ ] Visually verify `/dashboard` Recent Adverts card: route type badges appear, observer count badges appear, 10 rows shown, time column retained
- [ ] Visually verify `/advertisements` page: route type badges still render correctly (import refactor did not break them)
+94 -1
View File
@@ -14,12 +14,16 @@ from meshcore_hub.api.channel_visibility import (
resolve_user_role,
)
from meshcore_hub.api.dependencies import DbSession
from meshcore_hub.api.observer_utils import resolve_sender_names
from meshcore_hub.api.observer_utils import (
fetch_observers_for_events,
resolve_sender_names,
)
from meshcore_hub.common.models import (
Advertisement,
Message,
Node,
NodeTag,
RawPacket,
UserProfile,
)
from meshcore_hub.common.schemas.messages import (
@@ -147,6 +151,19 @@ def get_stats(
advertisements_24h = adv_row[1] or 0
advertisements_7d = adv_row[2] or 0
# Raw-packet counts (total + last 7 days), observer-level volume metric
# with no role/channel filter (payload redaction lives on list/detail only).
packet_row = session.execute(
select(
func.count(RawPacket.id).label("total_packets"),
func.sum(case((RawPacket.received_at >= seven_days_ago, 1), else_=0)).label(
"packets_7d"
),
).select_from(RawPacket)
).one()
total_packets = packet_row[0] or 0
packets_7d = packet_row[1] or 0
# Recent advertisements (last 10, flood-only)
recent_ads = (
session.execute(
@@ -185,6 +202,25 @@ def get_stats(
for public_key, value in session.execute(tag_name_query).all():
tag_names[public_key] = value
# Batch-resolve observers (via event_observers) and the legacy
# observed_by public key for the recent adverts, mirroring the
# advertisements list endpoint.
ad_event_hashes = [ad.event_hash for ad in recent_ads if ad.event_hash]
observers_by_hash = fetch_observers_for_events(
session, "advertisement", ad_event_hashes
)
observer_node_ids = [
ad.observer_node_id for ad in recent_ads if ad.observer_node_id
]
observer_pk_map: dict[str, str] = {}
if observer_node_ids:
obs_query = select(Node.id, Node.public_key).where(
Node.id.in_(observer_node_ids)
)
for node_id, public_key in session.execute(obs_query).all():
observer_pk_map[node_id] = public_key
recent_advertisements = [
RecentAdvertisement(
public_key=ad.public_key,
@@ -192,6 +228,13 @@ def get_stats(
tag_name=tag_names.get(ad.public_key),
adv_type=ad.adv_type or node_adv_types.get(ad.public_key),
received_at=ad.received_at,
route_type=ad.route_type,
observers=observers_by_hash.get(ad.event_hash, []) if ad.event_hash else [],
observed_by=(
observer_pk_map.get(ad.observer_node_id)
if ad.observer_node_id
else None
),
)
for ad in recent_ads
]
@@ -279,6 +322,8 @@ def get_stats(
channel_messages=channel_messages,
total_operators=total_operators,
total_members=total_members,
total_packets=total_packets,
packets_7d=packets_7d,
)
@@ -339,6 +384,54 @@ def get_activity(
return DailyActivity(days=days, data=data)
@router.get("/packet-activity", response_model=DailyActivity)
@cached("dashboard/packet-activity", ttl_setting="redis_cache_ttl_dashboard")
def get_packet_activity(
_: RequireRead,
session: DbSession,
request: Request,
days: int = 30,
) -> DailyActivity:
"""Get daily raw-packet activity for the specified period.
Args:
days: Number of days to include (default 30, max 90)
Returns:
Daily raw-packet counts for each day in the period (excluding today)
"""
days = min(days, 90)
now = datetime.now(timezone.utc)
end_date = now.replace(hour=0, minute=0, second=0, microsecond=0)
start_date = end_date - timedelta(days=days)
date_expr = func.date(RawPacket.received_at)
query = (
select(
date_expr.label("date"),
func.count().label("count"),
)
.where(RawPacket.received_at >= start_date)
.where(RawPacket.received_at < end_date)
.group_by(date_expr)
.order_by(date_expr)
)
results = session.execute(query).all()
counts_by_date = {_date_bucket_key(row.date): row.count for row in results}
data = []
for i in range(days):
date = start_date + timedelta(days=i)
date_str = date.strftime("%Y-%m-%d")
count = counts_by_date.get(date_str, 0)
data.append(DailyActivityPoint(date=date_str, count=count))
return DailyActivity(days=days, data=data)
@router.get("/message-activity", response_model=MessageActivity)
@cached(
"dashboard/message-activity",
@@ -242,6 +242,14 @@ class RecentAdvertisement(BaseModel):
tag_name: Optional[str] = Field(default=None, description="Name tag")
adv_type: Optional[str] = Field(default=None, description="Node type")
received_at: datetime = Field(..., description="When received")
route_type: Optional[str] = Field(default=None, description="Route type")
observers: list[ObserverInfo] = Field(
default_factory=list,
description="All observers that captured this advertisement",
)
observed_by: Optional[str] = Field(
default=None, description="Observing interface node public key"
)
class ChannelMessage(BaseModel):
@@ -288,6 +296,8 @@ class DashboardStats(BaseModel):
total_members: int = Field(
default=0, description="Number of member-role users (excluding operators)"
)
total_packets: int = Field(default=0, description="Total raw packets captured")
packets_7d: int = Field(default=0, description="Packets captured in last 7 days")
class DailyActivityPoint(BaseModel):
+16 -2
View File
@@ -30,6 +30,8 @@ const ChartColors = {
get advertsFill() { return withAlpha(this.adverts, 0.1); },
get messages() { return getCSSColor('--color-messages', 'oklch(0.75 0.18 180)'); },
get messagesFill() { return withAlpha(this.messages, 0.1); },
get packets() { return getCSSColor('--color-packets', 'oklch(0.72 0.17 145)'); },
get packetsFill() { return withAlpha(this.packets, 0.1); },
// Neutral grays (not page-specific)
grid: 'oklch(0.4 0 0 / 0.2)',
@@ -194,13 +196,14 @@ function createActivityChart(canvasId, advertData, messageData) {
}
/**
* Initialize dashboard charts (nodes, advertisements, messages).
* Initialize dashboard charts (nodes, advertisements, messages, packets).
* Pass null for any data parameter to skip that chart.
* @param {Object|null} nodeData - Node count data, or null to skip
* @param {Object|null} advertData - Advertisement data, or null to skip
* @param {Object|null} messageData - Message data, or null to skip
* @param {Object|null} packetData - Raw-packet data, or null to skip
*/
function initDashboardCharts(nodeData, advertData, messageData) {
function initDashboardCharts(nodeData, advertData, messageData, packetData) {
if (nodeData) {
createLineChart(
'nodeChart',
@@ -233,4 +236,15 @@ function initDashboardCharts(nodeData, advertData, messageData) {
true
);
}
if (packetData) {
createLineChart(
'packetChart',
packetData,
(window.t && window.t('entities.packets')) || 'Packets',
ChartColors.packets,
ChartColors.packetsFill,
true
);
}
}
@@ -185,6 +185,7 @@ export const pageColors = {
get nodes() { return getComputedStyle(document.documentElement).getPropertyValue('--color-nodes').trim(); },
get adverts() { return getComputedStyle(document.documentElement).getPropertyValue('--color-adverts').trim(); },
get messages() { return getComputedStyle(document.documentElement).getPropertyValue('--color-messages').trim(); },
get packets() { return getComputedStyle(document.documentElement).getPropertyValue('--color-packets').trim(); },
get map() { return getComputedStyle(document.documentElement).getPropertyValue('--color-map').trim(); },
get members() { return getComputedStyle(document.documentElement).getPropertyValue('--color-members').trim(); },
};
@@ -554,7 +555,20 @@ export function observerIcons(observers) {
if (!observers || observers.length === 0) return nothing;
const names = observers.map(o => o.tag_name || o.name || truncateKey(o.public_key, 8));
const tooltip = names.join(', ');
return html`<span class="badge badge-sm badge-primary cursor-help observer-badge" title=${tooltip}>${observers.length}</span>`;
return html`<span class="badge badge-sm badge-primary observer-badge" title=${tooltip}>${observers.length}</span>`;
}
export function routeTypeBadge(routeType) {
if (!routeType) {
return nothing;
}
if (routeType === 'flood' || routeType === 'transport_flood') {
return html`<span class="badge badge-sm badge-info">${routeType === 'flood' ? 'Flood' : 'Relay'}</span>`;
}
if (routeType === 'direct' || routeType === 'transport_direct') {
return html`<span class="badge badge-sm badge-success">${routeType === 'direct' ? 'Zero-hop' : 'Direct relay'}</span>`;
}
return nothing;
}
// --- Observer filter (localStorage-backed toggle badges) ---
@@ -802,10 +816,10 @@ export function renderFilterCard({ fields, basePath, navigate, submitLabel, clea
*/
export function renderStatCard({ icon, color, title, value, description }) {
return html`
<div class="stat bg-base-200 rounded-box shadow-sm panel-accent" style="--panel-color: ${color}">
<div class="stat bg-base-200 rounded-box shadow-sm panel-accent !py-2" style="--panel-color: ${color}">
<div class="stat-figure" style="color: ${color}">${icon}</div>
<div class="stat-title">${title}</div>
<div class="stat-value">${value}</div>
<div class="stat-value text-3xl">${value}</div>
${description ? html`<div class="stat-desc">${description}</div>` : nothing}
</div>`;
}
@@ -5,23 +5,10 @@ import {
warningBadge,
pagination, sortableTableHeader, mobileSortSelect,
renderFilterCard, autoSubmit, submitOnEnter, copyToClipboard, renderNodeDisplay,
observerIcons, getDisabledObservers, toggleObserver, observerFilterBadges
observerIcons, getDisabledObservers, toggleObserver, observerFilterBadges, routeTypeBadge
} from '../components.js';
import { createAutoRefresh } from '../auto-refresh.js';
function routeTypeBadge(routeType) {
if (!routeType) {
return nothing;
}
if (routeType === 'flood' || routeType === 'transport_flood') {
return html`<span class="badge badge-sm badge-info">${routeType === 'flood' ? 'Flood' : 'Relay'}</span>`;
}
if (routeType === 'direct' || routeType === 'transport_direct') {
return html`<span class="badge badge-sm badge-success">${routeType === 'direct' ? 'Zero-hop' : 'Direct relay'}</span>`;
}
return nothing;
}
export async function render(container, params, router) {
const { signal } = params || {};
const query = params.query || {};
@@ -2,10 +2,10 @@ import { apiGet, isAbortError } from '../api.js';
import {
html, litRender, nothing,
getConfig, getChannelLabelsMap, resolveChannelLabel,
typeEmoji, errorAlert, pageColors, renderStatCard, t, formatDateTime,
observerIcons, routeTypeBadge, errorAlert, t, formatDateTime,
} from '../components.js';
import {
iconNodes, iconAdvertisements, iconMessages, iconChannel,
iconNodes, iconAdvertisements, iconMessages, iconPackets, iconChannel,
} from '../icons.js';
function channelLabel(channel, channelLabels) {
@@ -35,12 +35,20 @@ function renderRecentAds(ads) {
if (!ads || ads.length === 0) {
return html`<p class="text-sm opacity-70">${t('common.no_entity_yet', { entity: t('entities.advertisements').toLowerCase() })}</p>`;
}
const rows = ads.slice(0, 5).map(ad => {
const rows = ads.map(ad => {
const friendlyName = ad.tag_name || ad.name;
const displayName = friendlyName || (ad.public_key.slice(0, 12) + '...');
const keyLine = friendlyName
? html`<div class="text-xs opacity-50 font-mono">${ad.public_key.slice(0, 12)}...</div>`
: nothing;
let observersBlock;
if (ad.observers && ad.observers.length >= 1) {
observersBlock = html`${observerIcons(ad.observers)}`;
} else if (ad.observed_by) {
observersBlock = html`<span class="opacity-50">\u{1F4E1}</span>`;
} else {
observersBlock = html`<span class="opacity-50">-</span>`;
}
return html`<tr>
<td>
<a href="/nodes/${ad.public_key}" class="link link-hover">
@@ -48,8 +56,9 @@ function renderRecentAds(ads) {
</a>
${keyLine}
</td>
<td>${ad.adv_type ? typeEmoji(ad.adv_type) : html`<span class="opacity-50">-</span>`}</td>
<td class="hidden md:table-cell">${routeTypeBadge(ad.route_type)}</td>
<td class="text-right text-sm opacity-70">${formatTimeOnly(ad.received_at)}</td>
<td>${observersBlock}</td>
</tr>`;
});
@@ -58,8 +67,9 @@ function renderRecentAds(ads) {
<thead>
<tr>
<th>${t('entities.node')}</th>
<th>${t('common.type')}</th>
<th class="hidden md:table-cell">${t('common.type')}</th>
<th class="text-right">${t('common.received')}</th>
<th>${t('common.observers')}</th>
</tr>
</thead>
<tbody>${rows}</tbody>
@@ -105,22 +115,30 @@ function renderChannelMessages(channelMessages, channelLabels) {
function gridCols(count) {
if (count === 2) return 'md:grid-cols-2';
if (count === 3) return 'md:grid-cols-3';
if (count === 4) return 'md:grid-cols-4';
return '';
}
function renderChartCards({ showNodes, showAdverts, showMessages }) {
const visibleCount = (showNodes ? 1 : 0) + (showAdverts ? 1 : 0) + (showMessages ? 1 : 0);
function renderChartCards({ showNodes, showAdverts, showMessages, showPackets, stats }) {
const visibleCount = (showNodes ? 1 : 0) + (showAdverts ? 1 : 0) + (showMessages ? 1 : 0) + (showPackets ? 1 : 0);
if (visibleCount === 0) return nothing;
return html`
<div class="grid grid-cols-1 ${gridCols(visibleCount)} gap-6 mb-8">
${showNodes ? html`
<div class="card bg-base-100 shadow-xl panel-accent" style="--panel-color: var(--color-nodes)">
<div class="card-body">
<h2 class="card-title text-base">
${iconNodes('h-5 w-5')}
${t('common.total_entity', { entity: t('entities.nodes') })}
</h2>
<p class="text-xs opacity-80">${t('time.over_time_last_7_days')}</p>
<div class="flex items-start justify-between gap-2">
<div>
<h2 class="card-title text-base">
${iconNodes('h-5 w-5')}
${t('entities.nodes')}
</h2>
<p class="text-xs opacity-80">${t('time.over_time_last_7_days')}</p>
</div>
<div class="text-4xl font-bold leading-none" style="color: var(--color-nodes)">
${stats.total_nodes}
</div>
</div>
<div class="h-32">
<canvas id="nodeChart"></canvas>
</div>
@@ -130,11 +148,18 @@ function renderChartCards({ showNodes, showAdverts, showMessages }) {
${showAdverts ? html`
<div class="card bg-base-100 shadow-xl panel-accent" style="--panel-color: var(--color-adverts)">
<div class="card-body">
<h2 class="card-title text-base">
${iconAdvertisements('h-5 w-5')}
${t('entities.advertisements')}
</h2>
<p class="text-xs opacity-80">${t('time.per_day_last_7_days')}</p>
<div class="flex items-start justify-between gap-2">
<div>
<h2 class="card-title text-base">
${iconAdvertisements('h-5 w-5')}
${t('entities.advertisements')}
</h2>
<p class="text-xs opacity-80">${t('time.per_day_last_7_days')}</p>
</div>
<div class="text-4xl font-bold leading-none" style="color: var(--color-adverts)">
${stats.advertisements_7d}
</div>
</div>
<div class="h-32">
<canvas id="advertChart"></canvas>
</div>
@@ -144,16 +169,44 @@ function renderChartCards({ showNodes, showAdverts, showMessages }) {
${showMessages ? html`
<div class="card bg-base-100 shadow-xl panel-accent" style="--panel-color: var(--color-messages)">
<div class="card-body">
<h2 class="card-title text-base">
${iconMessages('h-5 w-5')}
${t('entities.messages')}
</h2>
<p class="text-xs opacity-80">${t('time.per_day_last_7_days')}</p>
<div class="flex items-start justify-between gap-2">
<div>
<h2 class="card-title text-base">
${iconMessages('h-5 w-5')}
${t('entities.messages')}
</h2>
<p class="text-xs opacity-80">${t('time.per_day_last_7_days')}</p>
</div>
<div class="text-4xl font-bold leading-none" style="color: var(--color-messages)">
${stats.messages_7d}
</div>
</div>
<div class="h-32">
<canvas id="messageChart"></canvas>
</div>
</div>
</div>` : nothing}
${showPackets ? html`
<div class="card bg-base-100 shadow-xl panel-accent" style="--panel-color: var(--color-packets)">
<div class="card-body">
<div class="flex items-start justify-between gap-2">
<div>
<h2 class="card-title text-base">
${iconPackets('h-5 w-5')}
${t('entities.packets')}
</h2>
<p class="text-xs opacity-80">${t('time.per_day_last_7_days')}</p>
</div>
<div class="text-4xl font-bold leading-none" style="color: var(--color-packets)">
${stats.packets_7d}
</div>
</div>
<div class="h-32">
<canvas id="packetChart"></canvas>
</div>
</div>
</div>` : nothing}
</div>`;
}
@@ -166,12 +219,14 @@ export async function render(container, params, router) {
const showNodes = features.nodes !== false;
const showAdverts = features.advertisements !== false;
const showMessages = features.messages !== false;
const showPackets = features.packets !== false;
const [stats, advertActivity, messageActivity, nodeCount, channelsData] = await Promise.all([
const [stats, advertActivity, messageActivity, nodeCount, packetActivity, channelsData] = await Promise.all([
apiGet('/api/v1/dashboard/stats', {}, { signal }),
apiGet('/api/v1/dashboard/activity', { days: 7 }, { signal }),
apiGet('/api/v1/dashboard/message-activity', { days: 7 }, { signal }),
apiGet('/api/v1/dashboard/node-count', { days: 7 }, { signal }),
apiGet('/api/v1/dashboard/packet-activity', { days: 7 }, { signal }),
apiGet('/api/v1/channels', {}, { signal }),
]);
channelLabels = new Map([
@@ -181,10 +236,6 @@ export async function render(container, params, router) {
.filter(([idx]) => Number.isInteger(idx)),
]);
// Top section: stats + charts
const topCount = (showNodes ? 1 : 0) + (showAdverts ? 1 : 0) + (showMessages ? 1 : 0);
const topGrid = gridCols(topCount);
// Bottom section: recent adverts + recent channel messages
const bottomCount = (showAdverts ? 1 : 0) + (showMessages ? 1 : 0);
const bottomGrid = gridCols(bottomCount);
@@ -194,34 +245,8 @@ export async function render(container, params, router) {
<h1 class="text-3xl font-bold">${t('entities.dashboard')}</h1>
</div>
${topCount > 0 ? html`
<div class="grid grid-cols-1 ${topGrid} gap-6 mb-6">
${showNodes ? renderStatCard({
icon: iconNodes('h-8 w-8'),
color: pageColors.nodes,
title: t('common.total_entity', { entity: t('entities.nodes') }),
value: stats.total_nodes,
description: t('dashboard.all_discovered_nodes'),
}) : nothing}
${showAdverts ? renderStatCard({
icon: iconAdvertisements('h-8 w-8'),
color: pageColors.adverts,
title: t('entities.advertisements'),
value: stats.advertisements_7d,
description: t('time.last_7_days'),
}) : nothing}
${showMessages ? renderStatCard({
icon: iconMessages('h-8 w-8'),
color: pageColors.messages,
title: t('entities.messages'),
value: stats.messages_7d,
description: t('time.last_7_days'),
}) : nothing}
</div>
${renderChartCards({ showNodes, showAdverts, showMessages })}` : nothing}
${(showNodes || showAdverts || showMessages || showPackets) ? html`
${renderChartCards({ showNodes, showAdverts, showMessages, showPackets, stats })}` : nothing}
${bottomCount > 0 ? html`
<div class="grid grid-cols-1 ${bottomGrid} gap-6">
@@ -243,9 +268,10 @@ ${bottomCount > 0 ? html`
showNodes ? nodeCount : null,
showAdverts ? advertActivity : null,
showMessages ? messageActivity : null,
showPackets ? packetActivity : null,
);
const chartIds = ['nodeChart', 'advertChart', 'messageChart'];
const chartIds = ['nodeChart', 'advertChart', 'messageChart', 'packetChart'];
return () => {
chartIds.forEach(id => {
const canvas = document.getElementById(id);
@@ -35,7 +35,7 @@ function renderRadioTiles(rc) {
function renderNavCard({ href, icon, label, colorVar }) {
return html`
<a href="${href}" class="w-24 h-24 sm:w-28 sm:h-28
<a href="${href}" class="w-20 h-20 sm:w-[6.75rem] sm:h-[6.75rem]
border border-base-content/20 rounded-box
hover:scale-105 hover:border-base-content/40
transition-all duration-200 ease-out
@@ -75,7 +75,9 @@ function renderHeroSection({ networkName, logoUrl, logoInvertLight, networkCity,
<div class="flex-1 flex items-center justify-center w-full">
${welcomeText}
</div>
<div class="flex flex-wrap justify-center gap-2 sm:gap-3">
<div class="flex flex-wrap justify-center justify-items-center gap-2
sm:grid sm:grid-cols-4 min-[1536px]:grid-cols-8
sm:gap-3 min-[1536px]:gap-2">
${features.dashboard !== false ? renderNavCard({
href: '/dashboard',
icon: iconDashboard('w-full h-full'),
@@ -142,7 +144,7 @@ function renderStatsPanel({ features, stats }) {
${features.nodes !== false ? renderStatCard({
icon: iconNodes('h-8 w-8'),
color: pageColors.nodes,
title: t('common.total_entity', { entity: t('entities.nodes') }),
title: t('entities.nodes'),
value: stats.total_nodes,
description: t('home.all_discovered_nodes'),
}) : nothing}
@@ -160,6 +162,13 @@ function renderStatsPanel({ features, stats }) {
value: stats.messages_7d,
description: t('time.last_7_days'),
}) : nothing}
${features.packets !== false ? renderStatCard({
icon: iconPackets('h-8 w-8'),
color: pageColors.packets,
title: t('entities.packets'),
value: stats.packets_7d,
description: t('time.last_7_days'),
}) : nothing}
</div>`;
}
@@ -223,7 +232,7 @@ export async function render(container, params, router) {
apiGet('/api/v1/dashboard/message-activity', { days: 7 }, { signal }),
]);
const showStats = features.nodes !== false || features.advertisements !== false || features.messages !== false;
const showStats = features.nodes !== false || features.advertisements !== false || features.messages !== false || features.packets !== false;
const showAdvertSeries = features.advertisements !== false;
const showMessageSeries = features.messages !== false;
const showActivityChart = showAdvertSeries || showMessageSeries;
+217
View File
@@ -8,10 +8,12 @@ import pytest
from meshcore_hub.api.routes.dashboard import _date_bucket_key
from meshcore_hub.common.models import (
Advertisement,
EventObserver,
Message,
Node,
NodeTag,
Channel,
RawPacket,
)
from meshcore_hub.common.models import UserProfile
@@ -54,6 +56,8 @@ class TestDashboardStats:
assert data["total_messages"] == 0
assert data["messages_today"] == 0
assert data["total_advertisements"] == 0
assert data["total_packets"] == 0
assert data["packets_7d"] == 0
assert data["channel_message_counts"] == {}
def test_get_stats_with_data(
@@ -119,6 +123,15 @@ class TestDashboardStats:
),
]
)
# Raw packets: now (7d), 3 days ago (7d), 10 days ago (older).
api_db_session.add_all(
[
RawPacket(event_type="message", received_at=now),
RawPacket(event_type="message", received_at=now - timedelta(days=3)),
RawPacket(event_type="message", received_at=now - timedelta(days=10)),
]
)
api_db_session.commit()
data = client_no_auth.get("/api/v1/dashboard/stats").json()
@@ -134,6 +147,9 @@ class TestDashboardStats:
assert data["advertisements_24h"] == 1
assert data["advertisements_7d"] == 2 # now + 3d (10d excluded)
assert data["total_packets"] == 3
assert data["packets_7d"] == 2 # now + 3d (10d excluded)
class TestDashboardHtmlRemoved:
"""Tests that legacy HTML dashboard endpoint has been removed."""
@@ -269,6 +285,101 @@ class TestDashboardActivity:
assert yesterday_point["count"] >= 1
class TestPacketActivity:
"""Tests for GET /dashboard/packet-activity endpoint."""
def test_get_packet_activity_empty(self, client_no_auth):
"""Test getting packet activity with empty database."""
response = client_no_auth.get("/api/v1/dashboard/packet-activity")
assert response.status_code == 200
data = response.json()
assert data["days"] == 30
assert len(data["data"]) == 30
for point in data["data"]:
assert point["count"] == 0
assert "date" in point
def test_get_packet_activity_custom_days(self, client_no_auth):
"""Test getting packet activity with custom days parameter."""
response = client_no_auth.get("/api/v1/dashboard/packet-activity?days=7")
assert response.status_code == 200
data = response.json()
assert data["days"] == 7
assert len(data["data"]) == 7
def test_get_packet_activity_max_days(self, client_no_auth):
"""Test that packet activity is capped at 90 days."""
response = client_no_auth.get("/api/v1/dashboard/packet-activity?days=365")
assert response.status_code == 200
data = response.json()
assert data["days"] == 90
assert len(data["data"]) == 90
def test_get_packet_activity_with_data(self, client_no_auth, api_db_session):
"""Test getting packet activity with packets across two days.
Note: Activity endpoints exclude today's data to avoid showing
incomplete stats early in the day.
"""
now = datetime.now(timezone.utc)
yesterday = now - timedelta(days=1)
two_days_ago = now - timedelta(days=2)
observer = Node(
public_key="a" * 64,
name="Observer",
adv_type="REPEATER",
)
api_db_session.add(observer)
api_db_session.flush()
api_db_session.add_all(
[
RawPacket(
observer_node_id=observer.id,
event_type="message",
received_at=yesterday,
),
RawPacket(
observer_node_id=observer.id,
event_type="message",
received_at=yesterday,
),
RawPacket(
observer_node_id=observer.id,
event_type="message",
received_at=two_days_ago,
),
]
)
api_db_session.commit()
response = client_no_auth.get("/api/v1/dashboard/packet-activity?days=7")
assert response.status_code == 200
data = response.json()
assert data["days"] == 7
assert len(data["data"]) == 7
yesterday_str = yesterday.strftime("%Y-%m-%d")
two_days_ago_str = two_days_ago.strftime("%Y-%m-%d")
yesterday_point = next(p for p in data["data"] if p["date"] == yesterday_str)
assert yesterday_point["count"] == 2
two_days_ago_point = next(
p for p in data["data"] if p["date"] == two_days_ago_str
)
assert two_days_ago_point["count"] == 1
# All other days in the 7-day window should be zero-filled.
other_days = [
p
for p in data["data"]
if p["date"] not in (yesterday_str, two_days_ago_str)
]
assert all(p["count"] == 0 for p in other_days)
class TestMessageActivity:
"""Tests for GET /dashboard/message-activity endpoint."""
@@ -571,6 +682,9 @@ class TestDashboardFloodOnlyFilter:
data = response.json()
assert len(data["recent_advertisements"]) == 1
assert data["recent_advertisements"][0]["name"] == "Flood"
assert data["recent_advertisements"][0]["route_type"] == "flood"
assert data["recent_advertisements"][0]["observers"] == []
assert data["recent_advertisements"][0]["observed_by"] is None
def test_activity_excludes_direct(self, client_no_auth, api_db_session):
"""Activity endpoint excludes direct advertisements."""
@@ -768,6 +882,89 @@ class TestDashboardChannelVisibility:
data = response.json()
assert len(data["recent_advertisements"]) == 1
assert data["recent_advertisements"][0]["tag_name"] == "TagName"
assert data["recent_advertisements"][0]["route_type"] == "flood"
assert data["recent_advertisements"][0]["observers"] == []
assert data["recent_advertisements"][0]["observed_by"] is None
def test_recent_advertisements_includes_observers(
self, client_no_auth, api_db_session
):
"""Recent advertisements include observer list via event_observers."""
from hashlib import md5
now = datetime.now(timezone.utc)
observer_node = Node(
public_key="cc" * 16,
name="ObserverStation",
first_seen=now,
last_seen=now,
)
api_db_session.add(observer_node)
api_db_session.commit()
event_hash = md5(b"dashboard-ad-observers").hexdigest()
ad = Advertisement(
public_key="dd" * 16,
name="HeardAd",
adv_type="REPEATER",
received_at=now,
route_type="flood",
event_hash=event_hash,
observer_node_id=observer_node.id,
)
api_db_session.add(ad)
observer = EventObserver(
event_type="advertisement",
event_hash=event_hash,
observer_node_id=observer_node.id,
observed_at=now,
)
api_db_session.add(observer)
api_db_session.commit()
response = client_no_auth.get("/api/v1/dashboard/stats")
assert response.status_code == 200
data = response.json()
assert len(data["recent_advertisements"]) == 1
item = data["recent_advertisements"][0]
assert item["route_type"] == "flood"
assert len(item["observers"]) == 1
assert item["observers"][0]["public_key"] == observer_node.public_key
def test_recent_advertisements_includes_observed_by(
self, client_no_auth, api_db_session
):
"""Recent advertisements resolve observed_by from observer_node_id."""
now = datetime.now(timezone.utc)
observer_node = Node(
public_key="ee" * 16,
name="InterfaceNode",
first_seen=now,
last_seen=now,
)
api_db_session.add(observer_node)
api_db_session.commit()
ad = Advertisement(
public_key="ff" * 16,
name="LegacyAd",
adv_type="CLIENT",
received_at=now,
route_type="flood",
observer_node_id=observer_node.id,
)
api_db_session.add(ad)
api_db_session.commit()
response = client_no_auth.get("/api/v1/dashboard/stats")
assert response.status_code == 200
data = response.json()
assert len(data["recent_advertisements"]) == 1
item = data["recent_advertisements"][0]
assert item["route_type"] == "flood"
assert item["observers"] == []
assert item["observed_by"] == observer_node.public_key
def test_operator_sees_community_and_member_channel_counts(
self, client_no_auth, api_db_session
@@ -880,6 +1077,26 @@ class TestDashboardDateBucketRegression:
seeded_point = next(p for p in data["data"] if p["date"] == seeded_str)
assert seeded_point["count"] == 1
def test_packet_activity_nonzero_on_seeded_day(
self, client_no_auth, api_db_session
):
"""Packet activity chart shows non-zero count for the seeded day."""
two_days_ago = (datetime.now(timezone.utc) - timedelta(days=2)).replace(
hour=12, minute=0, second=0, microsecond=0
)
api_db_session.add(
RawPacket(
event_type="message",
received_at=two_days_ago,
)
)
api_db_session.commit()
data = client_no_auth.get("/api/v1/dashboard/packet-activity").json()
seeded_str = two_days_ago.strftime("%Y-%m-%d")
seeded_point = next(p for p in data["data"] if p["date"] == seeded_str)
assert seeded_point["count"] == 1
def test_node_count_steps_up_on_seeded_day(self, client_no_auth, api_db_session):
"""Node count chart steps up on the seeded creation day."""
two_days_ago = (datetime.now(timezone.utc) - timedelta(days=2)).replace(