Messages containing runs of newlines were preserved by the pre-wrap
styling on the messages view, stretching table rows and cards. Collapse
any run of newlines (and surrounding whitespace) into a single space at
display time, leaving stored text untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the #flash-banner flex centering rule to #system-banner so it matches
the network announcement layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add two operator-controlled, startup-time settings:
- SYSTEM_ANNOUNCEMENT: non-dismissable Markdown banner rendered above the
network announcement on every page (navbar -> system -> network).
- SYSTEM_MAINTENANCE: when enabled, forces all feature flags off so the nav
collapses to Home, hides the profile menu, and the SPA renders a maintenance
page for every route. The maintenance page makes no API calls, so the API
and database can be offline while the web component keeps running.
CLI exposes --system-announcement and tri-state --system-maintenance; the bool
falls back to pydantic settings to parse SYSTEM_MAINTENANCE reliably from env.
Adds i18n strings (en/nl), tests, and docs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the multi-select Observer dropdown buried in the filter panel with
a row of clickable observer badges rendered between the filter panel and the
data list (and below the Sorting dropdown on mobile).
- Selection is stored in localStorage (shared across Adverts and Messages)
as the disabled set, so new observers default to enabled.
- Badge style reflects enabled (filled) vs disabled (muted) state; the last
enabled observer cannot be toggled off.
- Observer filter is sourced from localStorage instead of the URL query;
a two-phase fetch resolves the enabled include-list (the API filters by
inclusion only) before fetching data, with no flash of unfiltered results.
- Toggling re-scopes data and resets to page 1.
- Add enable/disable tooltip strings (en/nl) and the previously-missing
Dutch observer label.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The observed_by filter on messages, advertisements, telemetry, and
trace_paths matched only the first observer (stored in observer_node_id),
silently excluding events whose secondary observers appear only in the
event_observers junction table. This caused filtered lists to appear
'several hours behind' when a dominant observer consistently won the
first-insert race for recent events.
Replace the ObserverNode.public_key predicate with an IN subquery against
the event_observers junction table (the canonical multi-observer source
already used for display). Add a shared observed_by_filter_clause() helper
in observer_utils.py to avoid duplication across all four routes.
Add regression tests proving a secondary observer (present only in
event_observers) sees events via the filter. Update existing fixtures and
inline test data to seed event_hash and EventObserver rows.
Fixes#239
Adverts/Messages rows now link directly to the deduplicated packet-detail
page (/packets/hash/:hash). Each path hop renders as a clickable badge
opening a popover that looks up nodes by public-key prefix via the new
pubkey_prefix query param on GET /api/v1/nodes (case-insensitive
startswith). Adds a derived path_hash_bytes field on GroupedPacketRead.
Defaults changed: FEATURE_PACKETS now defaults to true and
RAW_PACKET_RETENTION_DAYS to 7 (independent of DATA_RETENTION_DAYS).
Fixes a mypy arg-type error by explicitly annotating the packet_hash
list as list[str].
The packet-group detail observer table only ever showed the hop count
because _extract_path_hashes looked at decoded.payload.decoded.pathHashes,
but normal packets carry the routing path at the top level as decoded.path.
Read decoded.path first (falling back to the old pathHashes location).
Frontend: render each hop as its own badge joined by arrows, wrapping on
narrow screens, with centre-truncation for paths over 16 hops. Badges carry
a path-hash-badge class + data-path-hash hook for a future node lookup.
Also expose v1/packet-groups as an open endpoint in the web proxy mapping
(it is not a prefix of v1/packets) and rebase the packet_hash index
migration onto the current head.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Fix black formatting on packet_groups.py and raw_packets.py
- Fix mypy error: annotate sort_exprs dict as dict[str, Any] so
asc()/desc() receive a typed expression, not object
- Add test_packet_groups.py with 33 tests covering list/detail
endpoints, filters, redaction, observer hydration, path_hashes
extraction, and the _extract_path_hashes helper (98% line coverage)
https://claude.ai/code/session_01NH2rZzuHzasJj12SZeRhbJ
- packets.js: switch to /api/v1/packet-groups, replace observer cell
with reception badge (paths × observers), remove SNR/hops columns
and filters, navigate to /packets/hash/:hash on click
- packet-group-detail.js: new detail page showing all (observer, path)
receptions grouped by observer with path hash sequences
- app.js: register packetGroupDetail page and /packets/hash/:hash route
https://claude.ai/code/session_01NH2rZzuHzasJj12SZeRhbJ
- New GroupedPacketRead/PacketReceptionInfo/GroupedPacketList schemas
- GET /api/v1/packet-groups: two-phase GROUP BY query, 7-day default
window, lightweight Phase 2 (no raw_hex/decoded), role-aware cache
- GET /api/v1/packet-groups/{hash}: full reception list with path_hashes
extracted from decoded.payload.decoded.pathHashes
- New (packet_hash, received_at) composite index via Alembic migration
- i18n keys for receptions, path, observer counts
https://claude.ai/code/session_01NH2rZzuHzasJj12SZeRhbJ
Add a first-class Raw Packets feature that captures every inbound MeshCore
packet from the LetsMesh `packets` feed exactly as received, independent of
how the collector later classifies it.
Capture & storage
- New `RawPacket` model + migration (raw_packets table) with single and
composite indexes for the dominant filter-then-sort-by-newest queries.
- Collector-side `RAW_PACKET_CAPTURE_ENABLED` flag (default off); capture hook
reuses the decoder's per-hex cache (no second decode), one row per observer
reception, never blocks event dispatch.
- Separate `RAW_PACKET_RETENTION_DAYS` (falls back to DATA_RETENTION_DAYS);
cleanup runs regardless of capture so disabling drains the table. Raw-packet
observers retained in the is_observer recompute union.
API
- `GET /packets` and `/packets/{id}` with rich filtering, role-aware Redis
cache key, and channel-visibility redaction (restricted-channel packets are
returned metadata-only, not hidden, so pagination counts stay stable).
Web
- `FEATURE_PACKETS` flag (default off). Responsive Packets page (table desktop,
cards mobile) plus a Packet Detail page (breadcrumb nav, raw hex + decoded).
- Nav entry after Messages on all three surfaces; home.js reordered so Map
precedes Members; new packets icon + colour.
Finer-grained classification
- Replace the single `letsmesh_packet` catch-all with per-payload-type event
types (req, ack, encrypted_direct, encrypted_channel, grp_data, multipart,
control, raw_custom, ...); letsmesh_packet kept only as the unresolved-type
safety net.
Link from structured tables
- Add `packet_hash` to advertisements and messages (populated at ingest);
exact `packet_hash` filter on /packets; cube-icon link on the Adverts and
Messages lists -> /packets?packet_hash=<hash>, shown only when the feature is
on and the row has a stored hash.
Docs/config: .env.example, docker-compose (collector + web), AGENTS.md,
SCHEMAS.md, docs/letsmesh.md, docs/upgrading.md (## v0.13.0), en/nl i18n, and a
plan/tasks doc under docs/plans/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The profile page failed to render with "Cannot access 'h' before
initialization". The /profile/me branch destructured `signal` from
params and used it in the apiGet call, but later redeclared
`const signal = ac.signal` in the same block scope. Block-scoped const
hoisting put the earlier reference in the temporal dead zone.
Drop the redundant inner declaration and pass `ac.signal` directly to
the form submit listener.
Fixes#233
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The User Profile page rendered each adopted node's relative time from
adopted_at (the adoption date) rather than the node's most recent
activity, so it always showed the time since adoption (e.g. "35 days
ago") even when the node had advertised today.
Expose last_seen on AdoptedNodeRead and render it on the profile page,
falling back to "-" when null (matching the nodes/node-detail pages).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix dashboard pages stalling under rapid navigation, plus reduce the cost
of the heaviest dashboard endpoint.
SPA request cancellation: apiGet never passed an AbortSignal, so navigating
away left a page's in-flight requests running — the homepage alone fires
three (/stats + two charts), the slowest being /stats. Under rapid
navigation these piled up, holding browser connections and API threadpool
threads, so the page actually wanted queued behind stale work; a late
resolver could also clobber the new page's DOM.
- api.js: apiGet accepts an optional { signal } and forwards it to fetch;
export isAbortError().
- router.js: each navigation gets an AbortController; the previous one is
aborted at the start of _handleRoute and its signal is passed to the page
handler. A navigation-generation guard stops a superseded route from
hiding the loader for the page that replaced it.
- app.js: pageHandler swallows AbortError (an intentional cancel is not an
error).
- all 11 page modules: thread params.signal into on-load apiGet calls and
guard their catch blocks with isAbortError.
dashboard/stats consolidation: collapse the 11 sequential COUNT(*) queries
into 4 using portable conditional aggregation (func.sum(case(...))) for
nodes, messages, advertisements, and user profiles. Responses are
unchanged.
Docs: extend the v0.12 "Read-Path Query Optimisations" note and add a
"Dashboard Navigation Responsiveness" note (front-end only, no action
required).
Tests: add test_stats_time_bucket_counts asserting the active/today/24h/7d
buckets. SPA bundles are gitignored and rebuilt by the Docker/CI build, so
only committed source changed; the esbuild build was run locally to
validate the JS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a --workers / API_WORKERS option so the API can run multiple worker
processes in a single container for multi-core concurrency, without
needing to scale containers (which would conflict with the api service's
fixed container_name and complicate per-stack ops/monitoring).
The existing create_app() carries hardcoded defaults (sqlite:///./
meshcore.db, Redis off), so forked workers cannot use it — they would
open the wrong database and run without caching. Add an env-driven
factory, create_app_from_env(), that rebuilds the app from APISettings
plus the CLI-only env vars (CORS_ORIGINS, METRICS_ENABLED,
METRICS_CACHE_TTL), mirroring the single-process resolution. workers > 1
runs uvicorn against this factory via an import string; workers == 1
keeps the single-process object path so local CLI flags still apply.
Wire API_WORKERS into the api compose service (default 1, unchanged
behaviour) and document it in the README (new "Scaling the API"
section + env table) and the v0.12 upgrading notes, including the
SQLite single-host caveat and the env-vs-CLI-flag note for workers.
Tests: create_app_from_env reads DB/Redis/MQTT/metrics from env,
honours explicit overrides, and derives the collector DB path from
DATA_HOME rather than the bare create_app default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two read-only query optimisations, no schema changes.
node-count history: replace the per-day COUNT(*) loop (up to 90 full
scans of the unindexed created_at column) with two queries — a baseline
count of nodes created before the window plus one GROUP BY date()
aggregate, accumulated into the running total in Python. Results are
identical; the baseline seed keeps pre-window nodes counted from day 0.
sender-name resolution: add resolve_sender_names() to observer_utils,
batching all pubkey prefixes into two queries (names + name tags) via an
OR of indexable LIKE 'prefix%' terms instead of two queries per prefix.
Wire it into list_messages (was ~2xN per page) and the dashboard
channel-messages loop (nested per channel x per prefix). The dashboard
recent-ads block already batches on full public keys via IN(), so it is
left as-is.
Tests: add cumulative+baseline correctness for node-count and a
multi-sender batched-resolution case for messages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three related performance fixes for slow (>500ms) API responses.
1. Stop blocking the event loop. Route handlers were declared `async def`
but ran synchronous SQLAlchemy queries (and synchronous Redis calls via
the cache decorator) directly on the event loop, serializing requests.
Convert all handlers to sync `def` so FastAPI runs them in its
threadpool, and make the `@cached` decorator dual-mode (sync wrapper for
sync handlers, async wrapper preserved for a future async/Postgres path).
2. Tune SQLite for concurrency. Enable WAL, busy_timeout and
synchronous=NORMAL on every connection, and size the pool above the
threadpool so handlers don't wait on connections. In-memory SQLite is
guarded (no overflow-pool kwargs).
3. Precompute an indexed `nodes.is_observer` flag. The `observer=true`
filter scanned ~68k+ event rows to find a handful of observers (the
advertisements page calls it with limit=500). Replace the 5-way OR of
subqueries with `WHERE nodes.is_observer = ?`. The collector sets the
flag on first observation (in add_event_observer); the cleanup job
clears it once a node's events are all pruned; a migration adds the
column/index and backfills from the existing union.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace single NETWORK_RADIO_CONFIG comma-delimited string with six
individual environment variables: NETWORK_RADIO_PROFILE, _FREQUENCY,
_BANDWIDTH, _SPREADING_FACTOR, _CODING_RATE, _TX_POWER
- Radio config fields now use raw numeric types (float/int) with units
applied dynamically via RadioConfig.format_for_display()
- Add FEATURE_RADIO_CONFIG feature flag to control radio config panel
visibility on the home page (default: enabled)
- Remove from_config_string class method (no backwards compatibility)
- Update Click CLI options, create_app() signature, and _build_config_json()
- Update docker-compose.yml, .env.example, README.md, AGENTS.md
- Add upgrading.md v0.12.0 section with migration instructions
- Add test coverage for schema, config, and feature flag
- Rename ChannelVisibility.PUBLIC to ChannelVisibility.COMMUNITY
- Update stored value from 'public' to 'community' across model, schema, API, CLI, and frontend
- Add Alembic migration to update existing database rows
- Consolidate upgrade docs: merge v0.11.0, v0.12.0, v0.13.0 into single v0.11.0 section
- Add i18n visibility level translation keys (en, nl)
- Update section headings on channels page to use t() for i18n
- Keep visibility badges lowercase per UI design
- Group channel cards by visibility with section headings
- Move channels before messages in all nav menus for logical grouping
- Add optgroup labels (Standard/Custom) to message channel filter
- Capitalize built-in "Test" channel name for consistency
- Shorten "Advertisements" to "Adverts" in UI labels
- Lay out channel cards with side-by-side QR codes
- Shrink homepage nav cards for better fit
Replaces env-var channel keys with a Channel database model and periodic
DB refresh in the collector. Adds Channels dashboard page with QR codes,
channel visibility filtering on messages/dashboard APIs, and channel card
navigation to filtered messages view.
- Move route type badge left of observer count on mobile cards
- Hide route type badge when unknown instead of showing 'Unknown'
- Change observer count badge to badge-primary for visibility
- Remove satellite dish icon from observer count badge
Track advertisement route type (flood/transport_flood/direct/transport_direct)
and node advert timestamp to distinguish zero-hop from flood adverts, improve
deduplication with 300s buckets, and default all dashboard/ad-API queries to
flood-only (including NULL for historical records).
The node detail page sends a public_key query param when fetching
recent advertisements, but the API silently ignored it (FastAPI
ignores unknown query params). This returned the 10 most recent ads
network-wide instead of for the specific node, making all dates
appear as today.
The async SQLAlchemy engine was missing PRAGMA foreign_keys=ON, causing
ondelete="CASCADE" constraints to be silently ignored when the collector
deleted inactive nodes. This left orphaned rows in user_profile_nodes,
event_observers, and node_tags, which crashed the API with AttributeError
when accessing assoc.node.public_key on null relationships.
- Add FK PRAGMA listener to async engine (database.py)
- Add null-guard in _build_adopted_nodes() and refactor list_profiles()
- Add cleanup_orphaned_node_relations() covering all 3 dependent tables
- Integrate orphan cleanup into scheduled retention cycle (subscriber.py)
- Add --node-cleanup/--node-cleanup-days flags to CLI cleanup command
- Fix truncate cascade warning to include user_profile_nodes/event_observers
- Add FK PRAGMA to test fixtures for cascade verification
- Add upgrade note to docs/upgrading.md
- Fix Members page badge showing higher count than displayed profiles
by counting operators + members instead of all profiles
- Filter dropdowns on Nodes, Advertisements, and Map pages to show
only operators (since only operators can adopt nodes)
- Add roles field to /map/data profiles for client-side filtering
- Add all_operators and filter_operator_label i18n keys (en, nl)
- Fix flash banner test isolation from .env NETWORK_ANNOUNCEMENT
Add NETWORK_ANNOUNCEMENT env var that displays a dismissible flash banner
on every page when set. Announcement text supports Markdown (bold, italic,
links, inline code) rendered to HTML server-side at startup.
Add OIDC_ROLE_TEST config var (default: 'test') to exclude test users
from dashboard stats, member counts, and the Members page. Uses
server-side filtering with exclude_test query param (default: true) and
client-side defense-in-depth filter in members.js.
- Add oidc_role_test to WebSettings in config.py
- Exclude test users from operator/member count queries in dashboard.py
- Add exclude_test param to GET /api/v1/user/profiles in user_profiles.py
- Filter test users client-side in members.js via role_names.test config
- Wire oidc_role_test into app.state and frontend config in web/app.py
- Document OIDC_ROLE_TEST in AGENTS.md and .env.example
The web proxy's endpoint access mapping previously required one of
(admin/operator/member) roles for PUT /api/v1/user/profile. This
blocked OIDC users with no assigned roles from saving their own profile.
Add an _AUTHENTICATED sentinel access level that grants access to any
logged-in user regardless of roles, and apply it to the profile PUT
endpoint. The API layer already enforces owner-only checks via
RequireUserOwner, so the proxy role gate was redundant.
Remove stale members.yaml references (post-members-refactor), add
missing env vars to AGENTS.md (API_HOST, WEB_HOST, CORS_ORIGINS,
NETWORK_*), fix project structure tree, add backward-compat note for
MQTT_TOPIC_PREFIX, and update nl.json/i18n.md members section.
- Add .prose > :first-child margin-top:0 to eliminate double-spacing before headings
- Add nested list CSS (circle/square bullets, lower-alpha/roman numbering)
- Replace map popup <p>/space-y-1 layout with CSS grid for aligned label-value pairs
- Convert map filter card to collapsible <details> with state persistence
- Add nested list HTML output tests and markdown features docs
Replace the label:value list in the Network Info panel with a 3×2 grid of
compact tiles, each showing a themed cyan icon, label, and value. Adds 5 new
SVG icon functions and a --color-radio CSS custom property with dark/light variants.
Move mobile hamburger menu from navbar-start to navbar-end so it stays
right-aligned regardless of logo width. Mobile nav items are now rendered
via JS (renderMobileNav) using icon functions instead of duplicated inline
SVGs. Increase dropdown widths (w-52→w-56), icon sizes (h-4→h-5), and add
larger mobile tap targets via CSS media query.
- Change API and frontend default sort from name/asc to last_seen/desc
- Add mobileSortSelect() shared component for native select dropdown
- Add mobile sort select to nodes, advertisements, and messages pages
- Add i18n sort labels for all three list pages
- Update sort tests for new default with staggered timestamps
Bundle SPA JavaScript with esbuild for production builds, generating
content-hashed filenames for immutable caching. Vendor assets (Leaflet,
Chart.js, QRCode.js) get SHA256-based query params. Locale JSON files
get a combined hash version. Falls back to unbuiltsources when dist/
is absent.
Add sort/order query parameters to Nodes, Advertisements, and Messages
API endpoints. Nodes default to alpha-by-name (via COALESCE of name tag,
node name, public key). Ads and Messages default to newest-first.
Frontend adds sortableTableHeader() component with asc/desc toggle
indicators. Sort state is preserved in URL params, surviving
auto-refresh and pagination.
- Replace removed form-control/label-text/label-text-alt with DaisyUI v5
fieldset/fieldset-label equivalents in node-detail.js and map.js
- Switch JS error display from innerHTML+label-text-alt to
textContent+classList hidden toggling
- Redesign home hero navigation buttons as square cards with section
colors, border hover animation, and centered icons
- Move custom page links to a separate row with compact btn-outline style
- Remove deprecated btn-outline CSS overrides from app.css
- Add observer multi-select (<select multiple size=2>) to Advertisements and
Messages filter bars, populated from /api/v1/nodes?observer=true
- Make all filter sections collapsible via <details> on Nodes, Advertisements,
and Messages pages; collapsed by default, auto-expands when active filters
exist, preserves open state across auto-refresh ticks
- Add backend observer=true|false query param to GET /api/v1/nodes for
observer-only or non-observer-only node filtering via subquery
- Change observed_by in Advertisements/Messages API from single public_key
to list[str] with .in_() for multi-select support
- Fix router.js and api.js to handle array query params (duplicate keys
promoted to arrays, .append() per element)
- Fix createFilterHandler to use FormData.getAll() for multi-value support
- Replace DaisyUI form-control/label/label-text classes with Tailwind-native
equivalents (flex flex-col gap-1, flex items-center py-1, opacity-80 text-sm)
since DaisyUI CSS is tree-shaken from the build output
- Thicker collapsible border (border-2 border-base-content/25) visible in
both light and dark themes
- Bottom-align Filter/Clear buttons via two-row form layout
- Move Observer filter to last position on Advertisements page
- Add filter_observer_label i18n key
- Add tests for observer=true node filtering and multi observer params
- Replace nested <a> tags in member cards with @click handlers to fix
invalid HTML causing missing badge styles and double-navigation
race conditions (NS_BINDING_ABORT) on mobile
- Add e.preventDefault() to @click handlers to prevent parent <a>
activation
- Bump mobile dropdown z-index from z-[1] to z-50 so menu items
render above QR code wrapper (z-20) on node detail hero section
- Blur active element after SPA navigation to dismiss DaisyUI
:focus-within dropdown state
- Fix dashboard channel messages to show short date+time format
- Add overflow-x-auto to custom pages and node detail panels for
mobile overflow
- Fix tag editor: hide type column on mobile, truncate key/value
cells, show key in modal title instead of disabled input field
- Fix delete tag dialog using innerHTML for <strong> in i18n strings
- Fix tag add form not resetting after successful submit
- Add break-word and overflow-x-auto to prose tables in app.css
Replace the dedicated admin tag management page with inline tag editing
on the node detail page. Operators can now edit tags directly on nodes
they've adopted; admins retain unrestricted access.
Key changes:
- Remove admin SPA page (admin/index.js, admin/node-tags.js)
- Add inline tag editor to node-detail.js with add/edit/delete modals
- Replace RequireAdmin with RequireOperatorOrAdmin for tag API routes
- Add ownership check: operators restricted to adopted nodes only
- Add validate_and_coerce_tag_value for number/boolean coercion
- Remove unused bulk endpoints (copy, move, replace all)
- Use AbortController for event listeners to prevent accumulation
on lit-html DOM reuse across re-renders
- Track Leaflet map instance at module scope for defensive cleanup
- Fix checkAuthResponse to only redirect on 401 (not 403)
- Update tests for new OIDC-based auth model
- Update en.json locale, i18n.md, upgrading.md, AGENTS.md
Replace the role=infra NodeTag convention with UserProfileNode adoption
as the canonical infrastructure indicator across map, Prometheus metrics,
and alerting. Renames is_infra to is_adopted, infra_center to
adopted_center. Map icons change to blue (adopted) / green (normal),
with all adoption UI gated on OIDC_ENABLED. Adds meshcore_nodes_adopted
gauge and Alembic migration to clean up obsolete tags.
- Add Members button to homepage hero after Messages, using
--color-members palette for icon and border color
- Gated by features.members !== false (auto-disabled when
OIDC_ENABLED=false via existing server-side override)
- Move callsign badge inline with name in profile view,
matching Members card layout
Louis King