Restrict which remote observers may ingest events, keyed on the observer's
public key (the <public_key> segment of its LetsMesh upload topic). Anyone
with broker access can publish as an observer via JWT auth, so operators can
now gate ingestion.
- New ObserverFilter (case-insensitive prefix matching, allowlist overrides
denylist, accept-all when both empty)
- New OBSERVER_ALLOWLIST / OBSERVER_DENYLIST collector settings, wired through
the CLI, run_collector, create_subscriber, and Subscriber
- Filter applied at the top of _handle_mqtt_message: blocked observers' packets
are dropped before any decode, raw-packet capture, or DB write; zero added
work on the default accept-all path
- Tests: ObserverFilter unit tests, subscriber drop/allow integration tests,
config parsing tests
- Docs: configuration.md, observer.md, upgrading.md (v0.16.0), .env.example,
docker-compose.yml
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make FEATURE_SPAM_DETECTION on by default with opt-out, mirroring
FEATURE_PACKETS: flip the web feature flag's Python default to true and the
Compose substitutions (collector/api/web) to :-true, so the shipped stack
scores and hides likely-spam without configuration. Opt out with
FEATURE_SPAM_DETECTION=false.
Update .env.example, docs/configuration.md and the v0.15 upgrade notes to
describe the feature as enabled-by-default with opt-out.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adjust the default spam-scoring knobs across Python settings, SpamConfig,
docker-compose, .env.example and docs to reduce false positives on chatty
legitimate users:
SPAM_MIN_PATH_HOPS 5 -> 3
SPAM_PATH_THRESHOLD 5 -> 6
SPAM_NAME_THRESHOLD 5 -> 10
SPAM_WEIGHT_PATH 0.7 -> 0.75
SPAM_WEIGHT_NAME 0.3 -> 0.25
SPAM_SCORE_THRESHOLD 0.6 -> 0.65
Also document the spam-detection feature and the pull_policy change in a
new v0.15.0 section of docs/upgrading.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an optional, off-by-default spam-detection feature that scores each
message's spam likelihood at ingest, stores the score on the row, and lets
the display layer hide likely-spam by default behind a "show potential spam"
toggle. Nothing is ever dropped at ingest, so the threshold can be retuned
without reprocessing.
Scoring (collector/spam.py): windowed COUNT(*) over new
(path_prefix, received_at) and (sender_normalized, received_at) indexes —
joint path+sender signal plus a sender-name signal (trailing-digit suffix
stripped so bob1/bob2 collapse to bob). When the path is short/zero-hop or
absent, the name signal stands alone at full weight so local spam is still
flaggable. A background sweep re-scores recent rows with hindsight to catch
the leading edge of bursts. The collector logs each score (WARNING at/above
the threshold).
Display: the messages API gains include_spam and a master-switch-aware
hide-filter; the SPA shows the toggle + a badge only when the feature is on.
Config: FEATURE_SPAM_DETECTION is the single operator switch, bridged in
Compose to the backend SPAM_DETECTION_ENABLED for collector + api (mirrors
the FEATURE_PACKETS / RAW_PACKET_CAPTURE_ENABLED pattern). Both default off.
Works on SQLite and Postgres: DB-agnostic queries, an Alembic batch migration
for the three new columns + two indexes, and backend-aware collector test
fixtures (lifted db_backend/db_url into the shared conftest).
Also: move the meshcore-hub image pull_policy out of the base compose file.
It lived in docker-compose.yml as pull_policy: daily and made `make up` pull
the published image over a freshly built local one. Base is now policy-neutral
(default missing); dev sets pull_policy: build on the hub services so it only
ever uses local builds. Prod refreshes images via a manual `docker compose
... pull`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire the two new web settings into the web service env block, matching the
NETWORK_ANNOUNCEMENT passthrough. SYSTEM_MAINTENANCE defaults to false.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- compose: add optional postgres service (postgres:17-alpine, profile
'postgres', healthcheck, postgres_data volume); POSTGRES_* derive from
DATABASE_* (single source of truth). DATABASE_* env added to migrate/
collector/api; migrate depends_on postgres with required:false so SQLite
deployments are unaffected.
- alembic/env: resolve the URL via CommonSettings.effective_database_url so
DATABASE_BACKEND=postgres is honoured (previously DATABASE_URL/DATA_HOME
only -> would silently migrate SQLite).
- migrations: normalize_public_key uses STRING_AGG + HAVING COUNT(*) on
Postgres (was SQLite GROUP_CONCAT + alias); raw_packets uses sa.JSON()
not the sqlite dialect type.
- database: fix _to_async_url to map postgresql+psycopg2:// (what the config
assembles) to asyncpg, so API async sessions work on Postgres; resolve the
search_path schema from DATABASE_SCHEMA env when not passed explicitly.
Validated against a live postgres:17: db upgrade builds all 13 tables in the
meshcorehub schema with correct native types (is_observer boolean, decoded
json) and alembic_version stamped; the upsert, JSON/timestamptz round-trip,
and asyncpg async sessions all work. SQLite suite still green (1061 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adverts/Messages rows now link directly to the deduplicated packet-detail
page (/packets/hash/:hash). Each path hop renders as a clickable badge
opening a popover that looks up nodes by public-key prefix via the new
pubkey_prefix query param on GET /api/v1/nodes (case-insensitive
startswith). Adds a derived path_hash_bytes field on GroupedPacketRead.
Defaults changed: FEATURE_PACKETS now defaults to true and
RAW_PACKET_RETENTION_DAYS to 7 (independent of DATA_RETENTION_DAYS).
Fixes a mypy arg-type error by explicitly annotating the packet_hash
list as list[str].
Add a first-class Raw Packets feature that captures every inbound MeshCore
packet from the LetsMesh `packets` feed exactly as received, independent of
how the collector later classifies it.
Capture & storage
- New `RawPacket` model + migration (raw_packets table) with single and
composite indexes for the dominant filter-then-sort-by-newest queries.
- Collector-side `RAW_PACKET_CAPTURE_ENABLED` flag (default off); capture hook
reuses the decoder's per-hex cache (no second decode), one row per observer
reception, never blocks event dispatch.
- Separate `RAW_PACKET_RETENTION_DAYS` (falls back to DATA_RETENTION_DAYS);
cleanup runs regardless of capture so disabling drains the table. Raw-packet
observers retained in the is_observer recompute union.
API
- `GET /packets` and `/packets/{id}` with rich filtering, role-aware Redis
cache key, and channel-visibility redaction (restricted-channel packets are
returned metadata-only, not hidden, so pagination counts stay stable).
Web
- `FEATURE_PACKETS` flag (default off). Responsive Packets page (table desktop,
cards mobile) plus a Packet Detail page (breadcrumb nav, raw hex + decoded).
- Nav entry after Messages on all three surfaces; home.js reordered so Map
precedes Members; new packets icon + colour.
Finer-grained classification
- Replace the single `letsmesh_packet` catch-all with per-payload-type event
types (req, ack, encrypted_direct, encrypted_channel, grp_data, multipart,
control, raw_custom, ...); letsmesh_packet kept only as the unresolved-type
safety net.
Link from structured tables
- Add `packet_hash` to advertisements and messages (populated at ingest);
exact `packet_hash` filter on /packets; cube-icon link on the Adverts and
Messages lists -> /packets?packet_hash=<hash>, shown only when the feature is
on and the row has a stored hash.
Docs/config: .env.example, docker-compose (collector + web), AGENTS.md,
SCHEMAS.md, docs/letsmesh.md, docs/upgrading.md (## v0.13.0), en/nl i18n, and a
plan/tasks doc under docs/plans/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a --workers / API_WORKERS option so the API can run multiple worker
processes in a single container for multi-core concurrency, without
needing to scale containers (which would conflict with the api service's
fixed container_name and complicate per-stack ops/monitoring).
The existing create_app() carries hardcoded defaults (sqlite:///./
meshcore.db, Redis off), so forked workers cannot use it — they would
open the wrong database and run without caching. Add an env-driven
factory, create_app_from_env(), that rebuilds the app from APISettings
plus the CLI-only env vars (CORS_ORIGINS, METRICS_ENABLED,
METRICS_CACHE_TTL), mirroring the single-process resolution. workers > 1
runs uvicorn against this factory via an import string; workers == 1
keeps the single-process object path so local CLI flags still apply.
Wire API_WORKERS into the api compose service (default 1, unchanged
behaviour) and document it in the README (new "Scaling the API"
section + env table) and the v0.12 upgrading notes, including the
SQLite single-host caveat and the env-vs-CLI-flag note for workers.
Tests: create_app_from_env reads DB/Redis/MQTT/metrics from env,
honours explicit overrides, and derives the collector DB path from
DATA_HOME rather than the bare create_app default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace single NETWORK_RADIO_CONFIG comma-delimited string with six
individual environment variables: NETWORK_RADIO_PROFILE, _FREQUENCY,
_BANDWIDTH, _SPREADING_FACTOR, _CODING_RATE, _TX_POWER
- Radio config fields now use raw numeric types (float/int) with units
applied dynamically via RadioConfig.format_for_display()
- Add FEATURE_RADIO_CONFIG feature flag to control radio config panel
visibility on the home page (default: enabled)
- Remove from_config_string class method (no backwards compatibility)
- Update Click CLI options, create_app() signature, and _build_config_json()
- Update docker-compose.yml, .env.example, README.md, AGENTS.md
- Add upgrading.md v0.12.0 section with migration instructions
- Add test coverage for schema, config, and feature flag
Replaces env-var channel keys with a Channel database model and periodic
DB refresh in the collector. Adds Channels dashboard page with QR codes,
channel visibility filtering on messages/dashboard APIs, and channel card
navigation to filtered messages view.
Add NETWORK_ANNOUNCEMENT env var that displays a dismissible flash banner
on every page when set. Announcement text supports Markdown (bold, italic,
links, inline code) rendered to HTML server-side at startup.
NODE_CLEANUP_DAYS source of truth in Pydantic Settings is 30, not 7. Fixed in README, .env.example, docker-compose.yml, and docker-source-guide.md. Also added missing OIDC_POST_LOGOUT_REDIRECT_URI, WEB_AUTO_REFRESH_SECONDS, NETWORK_DOMAIN to docker-compose.yml web service. Added WEB_LOCALE and WEB_DATETIME_LOCALE to AGENTS.md env vars list.
Remove the static Member model/table, CRUD API, YAML seed files, and
admin UI. Replace with UserProfile-driven members page that reads roles
from OIDC identity provider. Key changes:
- Drop members table, add roles column to user_profiles (Alembic migration)
- Add GET /api/v1/user/profiles (paginated, no user_id exposed)
- Add GET /api/v1/user/profile/me (auto-creates profile for current user)
- Replace member_id node tag filter with adopted_by (profile UUID)
- Members page now shows profiles grouped by operator/member roles
- Profile page supports public view (/profile/:id) and owner edit (/profile)
- Node detail page shows adoption card side-by-side with public key card
- Auto-create user profile during OIDC login callback
- Hide Adopted Nodes section for non-operator/admin users
- Add member since date to profile cards
- Add role badges and adopted node badges to member tiles
- Add antenna/users icons to Members page group headers
- Rename compose profile 'receiver' -> 'observer' across all docs
- Rename docker-compose service 'packet-capture' -> 'observer' and volume
'packetcapture_data' -> 'observer_data'
- Change COMPOSE_PROJECT_NAME fallback from 'hub-dev' to 'hub' across all
compose files, Makefile, .env, docs
- Remove legacy interface-mock service from test compose file
- Remove monitoring services from all docker compose files — monitoring is now
user-managed infrastructure (users point their own Prometheus at /metrics)
- Remove metrics profile, prometheus/alertmanager volumes from Makefile
- Update README Getting Started: packet capture is included via --profile receiver,
not a separate prerequisite; add remote observers guide
- Add --profile all to all docker compose command examples in README and UPGRADING.md
- Simplify UPGRADING.md backup/migration to only meshcore_hub_data
Simplify the variable name to remove the legacy LetsMesh decoder prefix.
Also fix unparenthesized except tuples in web/app.py and promote the
parenthesized-exception rule to a prominent position in AGENTS.md.
Replace subprocess-based packet decoding with native Python meshcoredecoder>=0.3.2.
The decoder is now always enabled (no toggle), removing ENABLED/COMMAND/TIMEOUT
config vars. Adds _enrich_payload_decoded() to compensate for payload classes that
lack to_dict() overrides in the library. Removes Node.js/npm from Dockerfile and
deletes the patches/ directory. Adds parenthesized exception rule to AGENTS.md.
Remove the meshcore_interface component in favor of external
meshcore-packet-capture for data ingestion. Rename receiver_node_id
to observer_node_id across all models, schemas, handlers, and API
routes. Add Alembic migration for the column/table renames. Fix
frontend JS property name mismatch that prevented the Receiver column
from displaying observer data.
Integrate the meshcore-packet-capture image as an alternative to the
native interface-receiver, reorganize compose profiles so the receiver
profile uses packet capture while native-receiver covers the built-in
and mock receivers, and switch the default collector ingest mode to
letsmesh_upload.
Add /metrics endpoint with Prometheus gauges for nodes, messages,
advertisements, telemetry, trace paths, events, and members. Include
per-node last_seen timestamps for alerting. Add Alertmanager service
to Docker Compose metrics profile with default blackhole receiver.
Add NodeNotSeen alert rule (48h threshold). Add 1h time window to
all windowed metrics alongside existing 24h/7d/30d windows.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## i18n Refactoring
- Refactor admin translations to use common composable patterns
- Add common patterns: delete_entity_confirm, entity_added_success, move_entity_to_another_node, etc.
- Remove 18 duplicate keys from admin_members and admin_node_tags sections
- Update all admin JavaScript files to use new common patterns with dynamic entity composition
- Fix label consistency: rename first_seen to first_seen_label to match naming convention
## Translation Documentation
- Create comprehensive translation reference guide (languages.md) with 200+ documented keys
- Add translation architecture documentation to AGENTS.md with examples and best practices
- Add "Help Translate" call-to-action section in README with link to translation guide
- Add i18n feature to README features list
## Documentation Audit
- Add undocumented config options: API_KEY, WEB_LOCALE, WEB_DOMAIN to README and .env.example
- Fix outdated CLI syntax: interface --mode receiver → interface receiver
- Update database migration commands to use CLI wrapper (meshcore-hub db) instead of direct alembic
- Add static/locales/ directory to project structure section
- Add i18n configuration (WEB_LOCALE, WEB_THEME) to docker-compose.yml
## Testing
- All 438 tests passing
- All pre-commit checks passing (black, flake8, mypy)
- Added tests for new common translation patterns
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Operators can now disable specific pages (Dashboard, Nodes, Advertisements,
Messages, Map, Members, Pages) via FEATURE_* environment variables. Disabled
features are fully hidden: removed from navigation, return 404 on routes,
and excluded from sitemap/robots.txt. Dashboard auto-disables when all of
Nodes/Advertisements/Messages are off. Map auto-disables when Nodes is off.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add TZ environment variable support (standard Linux timezone)
- Create Jinja2 filters for timezone-aware formatting (localtime, localdate, etc.)
- Update all templates to use timezone filters with abbreviation suffix
- Pass TZ through docker-compose for web service
- Document TZ setting in README and AGENTS.md
Timestamps remain stored as UTC; only display is converted.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add YouTube channel URL configuration option alongside existing
GitHub/Discord/Email contact links. Also crop logo SVG to content
bounds and pass YouTube env var through docker-compose.
- Replace PAGES_HOME with CONTENT_HOME configuration (default: ./content)
- Content directory now contains pages/ and media/ subdirectories
- Add support for custom logo at $CONTENT_HOME/media/images/logo.svg
- Custom logo replaces favicon and navbar/home logos when present
- Mount media directory as /media for serving custom assets
- Simplify default logo to generic WiFi-style radiating arcs
- Update documentation and example directory structure
- Update tests for new CONTENT_HOME structure
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Allows adding static content pages (About, FAQ, etc.) as markdown files
with YAML frontmatter. Pages are stored in PAGES_HOME directory (default:
./pages), automatically appear in navigation menu, and are included in
the sitemap.
- Add PageLoader class to parse markdown with frontmatter
- Add /pages/{slug} route for rendering custom pages
- Add PAGES_HOME config setting to WebSettings
- Add prose CSS styles for markdown content
- Add pages to navigation and sitemap
- Update docker-compose.yml with pages volume mount
- Add comprehensive tests for PageLoader and routes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace node type badge with icon in admin tag editor
- Add Edit/Add Tags button on node detail page (when admin enabled and authenticated)
- Remove automatic seed container startup to prevent overwriting user changes
- Remove unused 'coordinate' value type from node tags (only string, number, boolean remain)
The web admin interface needs write permissions to create, update,
move, and delete node tags. Changed to use API_ADMIN_KEY with
fallback to API_READ_KEY if admin key is not configured.
Louis King