Add an optional, off-by-default spam-detection feature that scores each
message's spam likelihood at ingest, stores the score on the row, and lets
the display layer hide likely-spam by default behind a "show potential spam"
toggle. Nothing is ever dropped at ingest, so the threshold can be retuned
without reprocessing.
Scoring (collector/spam.py): windowed COUNT(*) over new
(path_prefix, received_at) and (sender_normalized, received_at) indexes —
joint path+sender signal plus a sender-name signal (trailing-digit suffix
stripped so bob1/bob2 collapse to bob). When the path is short/zero-hop or
absent, the name signal stands alone at full weight so local spam is still
flaggable. A background sweep re-scores recent rows with hindsight to catch
the leading edge of bursts. The collector logs each score (WARNING at/above
the threshold).
Display: the messages API gains include_spam and a master-switch-aware
hide-filter; the SPA shows the toggle + a badge only when the feature is on.
Config: FEATURE_SPAM_DETECTION is the single operator switch, bridged in
Compose to the backend SPAM_DETECTION_ENABLED for collector + api (mirrors
the FEATURE_PACKETS / RAW_PACKET_CAPTURE_ENABLED pattern). Both default off.
Works on SQLite and Postgres: DB-agnostic queries, an Alembic batch migration
for the three new columns + two indexes, and backend-aware collector test
fixtures (lifted db_backend/db_url into the shared conftest).
Also: move the meshcore-hub image pull_policy out of the base compose file.
It lived in docker-compose.yml as pull_policy: daily and made `make up` pull
the published image over a freshly built local one. Base is now policy-neutral
(default missing); dev sets pull_policy: build on the hub services so it only
ever uses local builds. Prod refreshes images via a manual `docker compose
... pull`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The observed_by filter on messages, advertisements, telemetry, and
trace_paths matched only the first observer (stored in observer_node_id),
silently excluding events whose secondary observers appear only in the
event_observers junction table. This caused filtered lists to appear
'several hours behind' when a dominant observer consistently won the
first-insert race for recent events.
Replace the ObserverNode.public_key predicate with an IN subquery against
the event_observers junction table (the canonical multi-observer source
already used for display). Add a shared observed_by_filter_clause() helper
in observer_utils.py to avoid duplication across all four routes.
Add regression tests proving a secondary observer (present only in
event_observers) sees events via the filter. Update existing fixtures and
inline test data to seed event_hash and EventObserver rows.
Fixes#239
Two read-only query optimisations, no schema changes.
node-count history: replace the per-day COUNT(*) loop (up to 90 full
scans of the unindexed created_at column) with two queries — a baseline
count of nodes created before the window plus one GROUP BY date()
aggregate, accumulated into the running total in Python. Results are
identical; the baseline seed keeps pre-window nodes counted from day 0.
sender-name resolution: add resolve_sender_names() to observer_utils,
batching all pubkey prefixes into two queries (names + name tags) via an
OR of indexable LIKE 'prefix%' terms instead of two queries per prefix.
Wire it into list_messages (was ~2xN per page) and the dashboard
channel-messages loop (nested per channel x per prefix). The dashboard
recent-ads block already batches on full public keys via IN(), so it is
left as-is.
Tests: add cumulative+baseline correctness for node-count and a
multi-sender batched-resolution case for messages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Rename ChannelVisibility.PUBLIC to ChannelVisibility.COMMUNITY
- Update stored value from 'public' to 'community' across model, schema, API, CLI, and frontend
- Add Alembic migration to update existing database rows
- Consolidate upgrade docs: merge v0.11.0, v0.12.0, v0.13.0 into single v0.11.0 section
- Add i18n visibility level translation keys (en, nl)
- Update section headings on channels page to use t() for i18n
- Keep visibility badges lowercase per UI design
Replaces env-var channel keys with a Channel database model and periodic
DB refresh in the collector. Adds Channels dashboard page with QR codes,
channel visibility filtering on messages/dashboard APIs, and channel card
navigation to filtered messages view.
Add sort/order query parameters to Nodes, Advertisements, and Messages
API endpoints. Nodes default to alpha-by-name (via COALESCE of name tag,
node name, public key). Ads and Messages default to newest-first.
Frontend adds sortableTableHeader() component with asc/desc toggle
indicators. Sort state is preserved in URL params, surviving
auto-refresh and pagination.
- Add observer multi-select (<select multiple size=2>) to Advertisements and
Messages filter bars, populated from /api/v1/nodes?observer=true
- Make all filter sections collapsible via <details> on Nodes, Advertisements,
and Messages pages; collapsed by default, auto-expands when active filters
exist, preserves open state across auto-refresh ticks
- Add backend observer=true|false query param to GET /api/v1/nodes for
observer-only or non-observer-only node filtering via subquery
- Change observed_by in Advertisements/Messages API from single public_key
to list[str] with .in_() for multi-select support
- Fix router.js and api.js to handle array query params (duplicate keys
promoted to arrays, .append() per element)
- Fix createFilterHandler to use FormData.getAll() for multi-value support
- Replace DaisyUI form-control/label/label-text classes with Tailwind-native
equivalents (flex flex-col gap-1, flex items-center py-1, opacity-80 text-sm)
since DaisyUI CSS is tree-shaken from the build output
- Thicker collapsible border (border-2 border-base-content/25) visible in
both light and dark themes
- Bottom-align Filter/Clear buttons via two-row form layout
- Move Observer filter to last position on Advertisements page
- Add filter_observer_label i18n key
- Add tests for observer=true node filtering and multi observer params
Remove the meshcore_interface component in favor of external
meshcore-packet-capture for data ingestion. Rename receiver_node_id
to observer_node_id across all models, schemas, handlers, and API
routes. Add Alembic migration for the column/table renames. Fix
frontend JS property name mismatch that prevented the Receiver column
from displaying observer data.
Internal database UUIDs (id, node_id, receiver_node_id) were being
exposed in API responses. These are implementation details that should
not be visible to API consumers. The canonical identifier for nodes
is the 64-char hex public_key.
Changes:
- Remove id, node_id from NodeTagRead, NodeRead schemas
- Remove id from MemberRead schema
- Remove id, receiver_node_id, node_id from MessageRead, AdvertisementRead,
TracePathRead, TelemetryRead schemas
- Update web map component to use public_key instead of member.id
for owner filtering
- Update tests to not assert on removed fields
- Update .flake8 and pre-commit config to properly use flake8 config
- Add B008 to ignored errors (FastAPI Depends pattern)
- Add E402 to ignored errors (intentional module-level imports)
- Remove unused imports from test files and source files
- Fix f-strings without placeholders
- Add type annotations to inner async functions
- Fix SQLAlchemy execute() to use text() wrapper
- Add type: ignore comments for alembic.command imports
- Exclude alembic/ directory from mypy in pre-commit
- Update mypy overrides for test files to not require type annotations
- Fix type annotations for params dicts in web routes
- Fix generator return type in test fixtures
- Add FastAPI application with lifespan management
- Implement bearer token authentication (read/admin levels)
- Create comprehensive REST API routes:
- Nodes: list, get by public key
- Node tags: CRUD operations
- Messages: list with filters, get by ID
- Advertisements: list with filters, get by ID
- Telemetry: list with filters, get by ID
- Trace paths: list with filters, get by ID
- Commands: send message, channel message, advertisement
- Dashboard: stats API and HTML dashboard
- Add API CLI command for running the server
- Create API test suite with 44 passing tests
Routes use proper RESTful status codes (201 Created, 204 No Content).
Authentication is optional - when keys not configured, endpoints are open.
Louis King