Adverts/Messages rows now link directly to the deduplicated packet-detail
page (/packets/hash/:hash). Each path hop renders as a clickable badge
opening a popover that looks up nodes by public-key prefix via the new
pubkey_prefix query param on GET /api/v1/nodes (case-insensitive
startswith). Adds a derived path_hash_bytes field on GroupedPacketRead.
Defaults changed: FEATURE_PACKETS now defaults to true and
RAW_PACKET_RETENTION_DAYS to 7 (independent of DATA_RETENTION_DAYS).
Fixes a mypy arg-type error by explicitly annotating the packet_hash
list as list[str].
Add a first-class Raw Packets feature that captures every inbound MeshCore
packet from the LetsMesh `packets` feed exactly as received, independent of
how the collector later classifies it.
Capture & storage
- New `RawPacket` model + migration (raw_packets table) with single and
composite indexes for the dominant filter-then-sort-by-newest queries.
- Collector-side `RAW_PACKET_CAPTURE_ENABLED` flag (default off); capture hook
reuses the decoder's per-hex cache (no second decode), one row per observer
reception, never blocks event dispatch.
- Separate `RAW_PACKET_RETENTION_DAYS` (falls back to DATA_RETENTION_DAYS);
cleanup runs regardless of capture so disabling drains the table. Raw-packet
observers retained in the is_observer recompute union.
API
- `GET /packets` and `/packets/{id}` with rich filtering, role-aware Redis
cache key, and channel-visibility redaction (restricted-channel packets are
returned metadata-only, not hidden, so pagination counts stay stable).
Web
- `FEATURE_PACKETS` flag (default off). Responsive Packets page (table desktop,
cards mobile) plus a Packet Detail page (breadcrumb nav, raw hex + decoded).
- Nav entry after Messages on all three surfaces; home.js reordered so Map
precedes Members; new packets icon + colour.
Finer-grained classification
- Replace the single `letsmesh_packet` catch-all with per-payload-type event
types (req, ack, encrypted_direct, encrypted_channel, grp_data, multipart,
control, raw_custom, ...); letsmesh_packet kept only as the unresolved-type
safety net.
Link from structured tables
- Add `packet_hash` to advertisements and messages (populated at ingest);
exact `packet_hash` filter on /packets; cube-icon link on the Adverts and
Messages lists -> /packets?packet_hash=<hash>, shown only when the feature is
on and the row has a stored hash.
Docs/config: .env.example, docker-compose (collector + web), AGENTS.md,
SCHEMAS.md, docs/letsmesh.md, docs/upgrading.md (## v0.13.0), en/nl i18n, and a
plan/tasks doc under docs/plans/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix dashboard pages stalling under rapid navigation, plus reduce the cost
of the heaviest dashboard endpoint.
SPA request cancellation: apiGet never passed an AbortSignal, so navigating
away left a page's in-flight requests running — the homepage alone fires
three (/stats + two charts), the slowest being /stats. Under rapid
navigation these piled up, holding browser connections and API threadpool
threads, so the page actually wanted queued behind stale work; a late
resolver could also clobber the new page's DOM.
- api.js: apiGet accepts an optional { signal } and forwards it to fetch;
export isAbortError().
- router.js: each navigation gets an AbortController; the previous one is
aborted at the start of _handleRoute and its signal is passed to the page
handler. A navigation-generation guard stops a superseded route from
hiding the loader for the page that replaced it.
- app.js: pageHandler swallows AbortError (an intentional cancel is not an
error).
- all 11 page modules: thread params.signal into on-load apiGet calls and
guard their catch blocks with isAbortError.
dashboard/stats consolidation: collapse the 11 sequential COUNT(*) queries
into 4 using portable conditional aggregation (func.sum(case(...))) for
nodes, messages, advertisements, and user profiles. Responses are
unchanged.
Docs: extend the v0.12 "Read-Path Query Optimisations" note and add a
"Dashboard Navigation Responsiveness" note (front-end only, no action
required).
Tests: add test_stats_time_bucket_counts asserting the active/today/24h/7d
buckets. SPA bundles are gitignored and rebuilt by the Docker/CI build, so
only committed source changed; the esbuild build was run locally to
validate the JS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a --workers / API_WORKERS option so the API can run multiple worker
processes in a single container for multi-core concurrency, without
needing to scale containers (which would conflict with the api service's
fixed container_name and complicate per-stack ops/monitoring).
The existing create_app() carries hardcoded defaults (sqlite:///./
meshcore.db, Redis off), so forked workers cannot use it — they would
open the wrong database and run without caching. Add an env-driven
factory, create_app_from_env(), that rebuilds the app from APISettings
plus the CLI-only env vars (CORS_ORIGINS, METRICS_ENABLED,
METRICS_CACHE_TTL), mirroring the single-process resolution. workers > 1
runs uvicorn against this factory via an import string; workers == 1
keeps the single-process object path so local CLI flags still apply.
Wire API_WORKERS into the api compose service (default 1, unchanged
behaviour) and document it in the README (new "Scaling the API"
section + env table) and the v0.12 upgrading notes, including the
SQLite single-host caveat and the env-vs-CLI-flag note for workers.
Tests: create_app_from_env reads DB/Redis/MQTT/metrics from env,
honours explicit overrides, and derives the collector DB path from
DATA_HOME rather than the bare create_app default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace single NETWORK_RADIO_CONFIG comma-delimited string with six
individual environment variables: NETWORK_RADIO_PROFILE, _FREQUENCY,
_BANDWIDTH, _SPREADING_FACTOR, _CODING_RATE, _TX_POWER
- Radio config fields now use raw numeric types (float/int) with units
applied dynamically via RadioConfig.format_for_display()
- Add FEATURE_RADIO_CONFIG feature flag to control radio config panel
visibility on the home page (default: enabled)
- Remove from_config_string class method (no backwards compatibility)
- Update Click CLI options, create_app() signature, and _build_config_json()
- Update docker-compose.yml, .env.example, README.md, AGENTS.md
- Add upgrading.md v0.12.0 section with migration instructions
- Add test coverage for schema, config, and feature flag
- Rename ChannelVisibility.PUBLIC to ChannelVisibility.COMMUNITY
- Update stored value from 'public' to 'community' across model, schema, API, CLI, and frontend
- Add Alembic migration to update existing database rows
- Consolidate upgrade docs: merge v0.11.0, v0.12.0, v0.13.0 into single v0.11.0 section
- Add i18n visibility level translation keys (en, nl)
- Update section headings on channels page to use t() for i18n
- Keep visibility badges lowercase per UI design
Replaces env-var channel keys with a Channel database model and periodic
DB refresh in the collector. Adds Channels dashboard page with QR codes,
channel visibility filtering on messages/dashboard APIs, and channel card
navigation to filtered messages view.
Track advertisement route type (flood/transport_flood/direct/transport_direct)
and node advert timestamp to distinguish zero-hop from flood adverts, improve
deduplication with 300s buckets, and default all dashboard/ad-API queries to
flood-only (including NULL for historical records).
The async SQLAlchemy engine was missing PRAGMA foreign_keys=ON, causing
ondelete="CASCADE" constraints to be silently ignored when the collector
deleted inactive nodes. This left orphaned rows in user_profile_nodes,
event_observers, and node_tags, which crashed the API with AttributeError
when accessing assoc.node.public_key on null relationships.
- Add FK PRAGMA listener to async engine (database.py)
- Add null-guard in _build_adopted_nodes() and refactor list_profiles()
- Add cleanup_orphaned_node_relations() covering all 3 dependent tables
- Integrate orphan cleanup into scheduled retention cycle (subscriber.py)
- Add --node-cleanup/--node-cleanup-days flags to CLI cleanup command
- Fix truncate cascade warning to include user_profile_nodes/event_observers
- Add FK PRAGMA to test fixtures for cascade verification
- Add upgrade note to docs/upgrading.md
Replace the dedicated admin tag management page with inline tag editing
on the node detail page. Operators can now edit tags directly on nodes
they've adopted; admins retain unrestricted access.
Key changes:
- Remove admin SPA page (admin/index.js, admin/node-tags.js)
- Add inline tag editor to node-detail.js with add/edit/delete modals
- Replace RequireAdmin with RequireOperatorOrAdmin for tag API routes
- Add ownership check: operators restricted to adopted nodes only
- Add validate_and_coerce_tag_value for number/boolean coercion
- Remove unused bulk endpoints (copy, move, replace all)
- Use AbortController for event listeners to prevent accumulation
on lit-html DOM reuse across re-renders
- Track Leaflet map instance at module scope for defensive cleanup
- Fix checkAuthResponse to only redirect on 401 (not 403)
- Update tests for new OIDC-based auth model
- Update en.json locale, i18n.md, upgrading.md, AGENTS.md
Replace the role=infra NodeTag convention with UserProfileNode adoption
as the canonical infrastructure indicator across map, Prometheus metrics,
and alerting. Renames is_infra to is_adopted, infra_center to
adopted_center. Map icons change to blue (adopted) / green (normal),
with all adoption UI gated on OIDC_ENABLED. Adds meshcore_nodes_adopted
gauge and Alembic migration to clean up obsolete tags.
Remove the static Member model/table, CRUD API, YAML seed files, and
admin UI. Replace with UserProfile-driven members page that reads roles
from OIDC identity provider. Key changes:
- Drop members table, add roles column to user_profiles (Alembic migration)
- Add GET /api/v1/user/profiles (paginated, no user_id exposed)
- Add GET /api/v1/user/profile/me (auto-creates profile for current user)
- Replace member_id node tag filter with adopted_by (profile UUID)
- Members page now shows profiles grouped by operator/member roles
- Profile page supports public view (/profile/:id) and owner edit (/profile)
- Node detail page shows adoption card side-by-side with public key card
- Auto-create user profile during OIDC login callback
- Hide Adopted Nodes section for non-operator/admin users
- Add member since date to profile cards
- Add role badges and adopted node badges to member tiles
- Add antenna/users icons to Members page group headers
- Pass client_id in logout redirect so LogTo can validate post_logout_redirect_uri
- Add OIDC_POST_LOGOUT_REDIRECT_URI config option with fallback derivation
- Move session.clear() after logout_redirect() to allow state data save
- Add 'username' to strip_userinfo() name fallback chain (LogTo uses this)
- Strip quotes from OIDC_SCOPES and pass as list to Authlib (fixes direnv
quoting issue where literal quotes were sent in the authorization URL)
- Add OIDC_POST_LOGOUT_REDIRECT_URI to config, app state, and docs
- Add INFO-level logging to callback and logout handlers for diagnostics
- Update .env.example, README.md, AGENTS.md, docs/upgrading.md
The LetsMesh normalizer stored public keys as UPPERCASE while the tag
importer stored them as lowercase, creating duplicate nodes for the same
device. Normalize all public keys to lowercase throughout:
- MQTT topic parsing (event, command, LetsMesh upload)
- LetsMesh normalizer output
- Node model __init__ enforcement
- Alembic migration to merge duplicates and normalize existing data
Louis King