SQLAlchemy 2.1 no longer installs greenlet by default, so the async
engine fails to import (Test, E2E api container), and its sources use
the `untyped-decorator` mypy code that the pinned mypy 1.9 hook rejects
(Lint).
- depend on sqlalchemy[asyncio] so greenlet is installed again
- bump the mirrors-mypy pre-commit hook to v2.4.0
- guard nullable tag values / channel_idx flagged by the newer mypy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFsp2i9E98EzovBaurmtEb
- Add IconRss component and feedHref prop to ListToolbar
- Wire feed links on Nodes, Advertisements, and Messages pages
- Messages feed link respects channel filter, defaults to idx 17
- Simplify advert/node feed item bodies, link adverts to packet hashes
- Add unit tests, page tests, and e2e specs for feed link behavior
Four feeds (public messages, node adverts deduped per node, new nodes,
per-community-channel messages), each in RSS 2.0 and Atom, built with
stdlib ElementTree. Feeds are hard-pinned to the logged-out view
(anonymous visibility, spam always filtered, identity headers ignored)
and served at clean public web URLs (/feeds/*) via a shared proxy helper
that forwards If-None-Match + X-Forwarded-* and gates on maintenance
mode. Responses are Redis-cached (ETag/304, public max-age) and
invalidated with messages/adverts/channels mutations. Item/self links
always target the public web endpoint; WEB_PUBLIC_URL pins the canonical
origin. Includes a FEATURE_FEEDS kill switch, SPA autodiscovery <link>
tags, and env/docs updates (v0.21.0 upgrading notes).
Remove the upgrade-compatibility items scheduled for v0.20 in the v0.19
release notes:
- the `meshcore-hub db migrate-to-postgres` command and its
`common/db_migrate.py` module (with its dedicated tests)
- the `database_backend` settings field and its legacy-backend validator
(a leftover DATABASE_BACKEND is now silently ignored)
- the `[postgres]` pip extra alias (Dockerfile installs `.` now)
`tests/test_main.py` is rewritten as a minimal CLI smoke test to keep
entry-point coverage. Docs updated: v0.20.0 upgrade section notes the
removals (SQLite deployments must upgrade through v0.19 first), README
and database.md pointers reworded, stale .env.example comment dropped.
Redis becomes mandatory infrastructure (PostgreSQL-parity): the
REDIS_ENABLED flag, NullCache backend, and every conditional caching
code path are removed. /health/ready reports not_ready (503) when
Redis is unreachable and cached endpoints return 503 instead of
silently degrading; invalidation helpers never raise after a
committed write.
The bundled redis container joins the core profile only via
docker-compose.dev.yml (production points REDIS_* at an external
instance, same no-shadowing pattern as postgres), the backend suite
runs against real Redis (test-redis on 127.0.0.1:55433, CI redis:8
service container, per-xdist-worker key prefixes), and operator docs
describe the required-dependency setup (upgrading.md gains the
v0.20.0 section).
Marker clustering changed the map DOM contract the spec relied on:
clustered and off-viewport markers are absent, so raw .map-marker counts
undercount and first-marker clicks can hit-test against the sticky
navbar/banners. Count nodes as individual markers plus cluster badge
totals (new data-testid on the badge), zoom into a cluster before the
label and popup interactions so targets sit mid-map, and assert cluster
clicks spread children into individual markers.
Wrap MapPage node markers in react-leaflet-cluster (leaflet.markercluster)
with spiderfy-at-max-zoom, zoom-to-bounds on click, clustering disabled at
zoom >= 15, and viewport culling so off-screen markers stay out of the DOM.
Cluster bubbles render a plain themed count badge. Marker label HTML is now
only built when the show-labels toggle is on, halving per-marker DOM weight;
hover-to-reveal no longer applies while labels are toggled off.
- channels: post-delete card count is 6 (6 seeded + 1 created - 1), not 5
- empty-states/pagination: the empty text renders twice (mobile
EmptyState + desktop EmptyRow); scope assertions to the table cell
- mobile: list-row is desktop-only; assert the mobile sort select and
card text instead
- not-found: '/$' is a glob, not a regex; assert the exact URL '/'
- packets-detail: the newest seeded packet is the ce01 message, not an
advert; assert the event type returned by the API
Grow the suite from 41 to 77 specs covering previously untested UI
surfaces: channels admin CRUD with QR codes and keyboard nav, node
detail adoption/tag editing with the role permission matrix, SPA 404
page, sortable tables with URL-synced sort state, pagination with
filter-preserving links, mobile viewport (hamburger nav, mobile cards,
sort select), anonymous/member auth gating, raw packet detail, and
empty states. Route cards now assert history chart, recent-match links
and owner profile links.
Supporting changes: purposeful data-testids on channels/node-detail/
pagination/sort headers (plus aria-sort) and mobile nav; seed_data adds
30 trace_data filler groups for pagination, channels at every
visibility tier incl. one disabled, and a pw-operator profile.
Add a summary job that runs after all CI jobs (even on failure),
parses the pytest/vitest/Playwright JUnit reports plus coverage.xml,
and renders a consolidated Markdown summary to $GITHUB_STEP_SUMMARY:
job results, test counts, durations, backend coverage, failure
details, and the artifact inventory. Enable a Playwright JUnit
reporter and include it in the playwright-report artifact.
Starting core with docker-compose.prod.yml also started the bundled
postgres container, whose 'postgres' network alias shadowed a shared
instance of the same name on the same Docker network — the live site
silently talked to an empty local database instead of DATABASE_HOST.
- Base file: postgres profiles are now [all, postgres] (core dropped);
the dev override unions 'core' back in, so the documented dev
workflow (make up, --profile core) is unchanged
- migrate's postgres dependency is required: false — still health-gated
when the bundled container is active, skipped when profile-inactive
- prod override documents the behavior; opt in explicitly with
--profile postgres (or --profile all) for a bundled production DB
- Docs: database.md/deployment.md/configuration.md/README/.env.example
updated; upgrading.md gains a one-time cleanup runbook (full down to
remove the previously started hub-postgres container)
- /health/ready returns 503 (not 200) when not ready; body no longer leaks
raw backend error text (api + web tiers)
- SYSTEM_MAINTENANCE now gates the web /api/* proxy (503 MAINTENANCE)
instead of only hiding the SPA UI
- channel mutations invalidate messages/packets/packet_groups/dashboard
caches so visibility/key changes apply immediately (was stale up to 1h)
- async DB engine: pool_size/max_overflow/pre_ping matching the sync
engine (survives Postgres restarts); adispose() awaited from the API
lifespan (async connections were leaked by the sync dispose)
- webhooks: permanent 4xx (except 408/429) fail fast instead of retrying;
dispatch queue bounded at 1000 with drop-oldest + warning; shutdown
logs dropped events; matches_event evaluated once
- SPA: apiGet/apiPostForm redirect to OIDC login on 401; chart date
labels and route timestamps rendered in UTC/configured timezone
- e2e compose collector healthcheck uses 'meshcore-hub health collector'
(image has no pgrep)
- docs/upgrading.md: fold unreleased notes into v0.18.0 and document the
above
- api CLI single-process path now passes spam_detection_enabled /
spam_score_threshold to create_app() so the messages hide-filter
actually runs in the default (Docker) deployment
- OIDC role names (oidc_role_admin/operator/member/test) moved from
WebSettings to CommonSettings and wired onto API app.state; the API
tier previously matched hardcoded role names, ignoring customized
OIDC_ROLE_* config in authz, channel visibility, and tag/adoption
checks (oidc_roles_claim stays web-only)
- /metrics handler is now sync def so collection runs in the
threadpool instead of blocking the event loop; hoisted fragile
in-loop imports
- evaluate_route_history gates max_path_length on the packet's true
hop count (RawPacket.path_len) instead of the prefix-filtered hop
list, so history day buckets agree with snapshot evaluation
- reload_keys clears the decode cache: negatively-cached packets for
a newly added channel key were permanently dropped by the normalizer
Also documents 'sudo npx playwright install-deps chromium' in the
AGENTS.md e2e setup.
- /map/data: resolve viewer OIDC session and apply the endpoint access
mapping before fetching profiles; serve Cache-Control: private when
OIDC is enabled (payload varies by viewer)
- /user/profile/me + /user/profile/{id}: trust X-User-Id only with a
valid API key (new optional_user_identity dependency); closes the
OIDC subject-identifier leak via forged headers
- OIDC_SESSION_SECRET is now mandatory when OIDC is enabled (hard fail,
no insecure-dev-secret fallback)
- /metrics denies unauthenticated access when no API read key is set;
new METRICS_PUBLIC opt-in (Settings, CLI, compose, env factory)
- CORS: unset CORS_ORIGINS keeps wildcard compat but drops
allow_credentials (invalid combination in browsers) + warns
- Web tier: SecurityHeadersMiddleware (nosniff, X-Frame-Options DENY,
Referrer-Policy, Permissions-Policy) + nonce-based CSP; nonce applied
to spa.html inline scripts; WEB_SECURITY_HEADERS kill switch and
WEB_CSP_EXTRA for custom tile hosts; API emits nosniff
Breaking changes documented in docs/upgrading.md.
The map endpoint fetched a single limit=500 page (the API's hard max,
sorted last_seen desc), so networks with more than 500 nodes silently
hid stale/offline nodes from the unfiltered map — including adopted
infrastructure, which still appeared under the adopted_by filter
because that filter narrows the query before the limit applies.
Follow offset/total pagination via a new _fetch_all_pages helper (40
pages / 20k nodes safety cap, partial results on later-page failure)
for both /api/v1/nodes and /api/v1/user/profiles, and report the
API-reported total in debug.total_nodes.
The meshcore-mqtt-broker destroys subscriber connections (keep-alive
timeout) without sending FIN/RST, leaving a half-open socket that paho
never sees as disconnected: the collector stays "healthy" but silently
stops ingesting forever.
- Enable TCP keepalive (idle 60s / intvl 30s / count 3) on every
connect, unwrapping paho's _WebsocketWrapper to reach the raw socket;
kernel now fails zombie sockets so paho reconnects and resubscribes
- Verify SUBACKs via on_subscribe; log broker refusals (0x80) as ERROR
- Resubscribe with the registered QoS instead of paho's QoS 0 default
- Drop the optimistic _mqtt_connected flag; health now reports
mqtt.is_connected, driven by real CONNACK/disconnect callbacks
PR #348 changed several defaults but left a handful of docs, the
docker-compose fallback, example seed files, and code docstrings stale:
- docker-compose.yml: RAW_PACKET_RETENTION_DAYS fallback 7->2 (the
:-7 override was the most consequential — operators without the env
var set got 7 days, contradicting the code default and .env.example)
- docs/configuration.md: route-history backfill now references the
300 s evaluator cadence (was 60 s)
- docs/seeding.md + example/seed/routes.yaml: window_hours 48/24 -> 6
(matches the new default; both exceeded the 12 h max)
- routes.py docstring: evaluator tick 60 s -> 300 s
- subscriber.py: raw_packet_retention_days signature defaults 7 -> 2
(overridden at runtime by effective_raw_packet_retention_days, but
kept consistent with the authoritative config default)
PR #348 lowered the RAW_PACKET_RETENTION_DAYS default 7->2, but the
dashboard packet count, activity chart, and breakdown chart still
advertised "Last 7 days" — a label that lied once only 2 days of data
remained. Make every raw-packet-derived window track the effective
retention so labels stay honest:
- get_dashboard_stats: packets_7d window = min(7, retention); expose
packets_window_days on DashboardStats so the UI renders the real window
- packet-activity / packet-breakdown: clamp days = min(days, 90, retention)
so responses never advertise days whose data has been purged
- Home StatCard + Dashboard chart subtitles: dynamic "Last N days" /
"Per day (last N days)" via new time.last_n_days / time.per_day_last_n_days
i18n keys ({{n}} interpolation, matching routes_over_last_n_days)
- ActivitySeries.days made optional (type shared across activity charts);
PacketBreakdown.days added (backend always serializes it)
Messages/ads 7-day displays are unchanged — those draw on event data
which retains the full 30-day DATA_RETENTION_DAYS window.
Tests: autouse retention fixture (->90) on the three packet test classes
so existing days-handling tests stay valid; added retention-clamp tests
proving the window tracks retention=2 and relaxes to 7 at retention>=7.
JingleManSweep