FastAPI 0.137.0 refactored include_router to keep included routers as
nested objects rather than flattening their routes into app.routes, so
test_app_factory's `{route.path for route in app.routes}` no longer found
the /metrics route (the endpoint still serves; only this introspection
broke). FastAPI now treats router.routes as an internal implementation
detail.
Switch the metrics route checks to the public OpenAPI schema
(app.openapi()["paths"]), which is stable across versions and resolves
router prefixes correctly, and drop the <0.137.0 pin that was blocking
the upgrade.
Verified: app factory tests pass on both 0.136.3 and 0.137.2; full
tests/test_api suite (462 tests) passes on 0.137.2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restrict which remote observers may ingest events, keyed on the observer's
public key (the <public_key> segment of its LetsMesh upload topic). Anyone
with broker access can publish as an observer via JWT auth, so operators can
now gate ingestion.
- New ObserverFilter (case-insensitive prefix matching, allowlist overrides
denylist, accept-all when both empty)
- New OBSERVER_ALLOWLIST / OBSERVER_DENYLIST collector settings, wired through
the CLI, run_collector, create_subscriber, and Subscriber
- Filter applied at the top of _handle_mqtt_message: blocked observers' packets
are dropped before any decode, raw-packet capture, or DB write; zero added
work on the default accept-all path
- Tests: ObserverFilter unit tests, subscriber drop/allow integration tests,
config parsing tests
- Docs: configuration.md, observer.md, upgrading.md (v0.16.0), .env.example,
docker-compose.yml
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The spam-detection change wrapped the message table cell content in a
multi-line div/span so the spam badge could sit alongside the text, but left
white-space: pre-wrap on the td. The template literal's own indentation
(newlines + spaces around the div) then rendered as literal whitespace inside
the pre-wrap cell, padding out every row regardless of spam state.
Move pre-wrap onto the span that holds the message text so multi-line bodies
still wrap while the cell's layout whitespace collapses normally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make FEATURE_SPAM_DETECTION on by default with opt-out, mirroring
FEATURE_PACKETS: flip the web feature flag's Python default to true and the
Compose substitutions (collector/api/web) to :-true, so the shipped stack
scores and hides likely-spam without configuration. Opt out with
FEATURE_SPAM_DETECTION=false.
Update .env.example, docs/configuration.md and the v0.15 upgrade notes to
describe the feature as enabled-by-default with opt-out.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Messages page badged a row as spam at a hardcoded score >= 0.6 while the
API hides rows at SPAM_SCORE_THRESHOLD (now 0.65), so messages scored in
[0.6, 0.65) appeared flagged but were never hidden by the "show potential
spam" toggle.
Expose spam_score_threshold in the SPA config and use it for the badge so a
row is badged exactly when the API would hide it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adjust the default spam-scoring knobs across Python settings, SpamConfig,
docker-compose, .env.example and docs to reduce false positives on chatty
legitimate users:
SPAM_MIN_PATH_HOPS 5 -> 3
SPAM_PATH_THRESHOLD 5 -> 6
SPAM_NAME_THRESHOLD 5 -> 10
SPAM_WEIGHT_PATH 0.7 -> 0.75
SPAM_WEIGHT_NAME 0.3 -> 0.25
SPAM_SCORE_THRESHOLD 0.6 -> 0.65
Also document the spam-detection feature and the pull_policy change in a
new v0.15.0 section of docs/upgrading.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Guard the optional `_spam_rescore_thread` with an `is not None` assert before
calling `.is_alive()`, matching the sibling test. CI runs `pre-commit run
--all-files`, so mypy checks the test files too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Raise diff/patch coverage above the Codecov gate (was 79.8%, target 80.29%)
by exercising the previously-untested spam paths:
- subscriber: TestSpamRescoreScheduler covers the disabled early-return, the
enabled thread spawning + one sweep + clean stop, and the swallowed-error
branch of the background re-scoring loop.
- handler: a contact-message scoring test covers the contact log branch.
- spam: get/reset_spam_config caching, the zero-weight combine path, the
default-`now` path, and the null-sender reset in rescore_recent.
Patch coverage for the change is now ~97%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an optional, off-by-default spam-detection feature that scores each
message's spam likelihood at ingest, stores the score on the row, and lets
the display layer hide likely-spam by default behind a "show potential spam"
toggle. Nothing is ever dropped at ingest, so the threshold can be retuned
without reprocessing.
Scoring (collector/spam.py): windowed COUNT(*) over new
(path_prefix, received_at) and (sender_normalized, received_at) indexes —
joint path+sender signal plus a sender-name signal (trailing-digit suffix
stripped so bob1/bob2 collapse to bob). When the path is short/zero-hop or
absent, the name signal stands alone at full weight so local spam is still
flaggable. A background sweep re-scores recent rows with hindsight to catch
the leading edge of bursts. The collector logs each score (WARNING at/above
the threshold).
Display: the messages API gains include_spam and a master-switch-aware
hide-filter; the SPA shows the toggle + a badge only when the feature is on.
Config: FEATURE_SPAM_DETECTION is the single operator switch, bridged in
Compose to the backend SPAM_DETECTION_ENABLED for collector + api (mirrors
the FEATURE_PACKETS / RAW_PACKET_CAPTURE_ENABLED pattern). Both default off.
Works on SQLite and Postgres: DB-agnostic queries, an Alembic batch migration
for the three new columns + two indexes, and backend-aware collector test
fixtures (lifted db_backend/db_url into the shared conftest).
Also: move the meshcore-hub image pull_policy out of the base compose file.
It lived in docker-compose.yml as pull_policy: daily and made `make up` pull
the published image over a freshly built local one. Base is now policy-neutral
(default missing); dev sets pull_policy: build on the hub services so it only
ever uses local builds. Prod refreshes images via a manual `docker compose
... pull`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move scattered configuration tables and operational sections out of the
README into dedicated reference documents:
- docs/configuration.md: single source of truth for all environment
variables, grouped into 12 sections (Common, Database, Caching,
Collector, Webhooks, Auth, Data Retention, API, Web Dashboard,
Feature Flags, Traefik, Prometheus & Alertmanager)
- docs/deployment.md: production setup, reverse proxy, multi-instance,
API scaling, Redis caching
- docs/observer.md: remote observers plus PACKETCAPTURE_* and
SERIAL_PORT reference
- docs/maintenance.md: backup and restore
README is reduced from 712 to 385 lines; the ARM32/Raspberry Pi note
is dropped. database.md, auth.md, webhooks.md, and content.md have
their env-var tables removed and link back to configuration.md. Stale
cross-references in database.md, upgrading.md, and .env.example are
updated to point at the new locations.
Remove paths-ignore from the pull_request trigger so the CI workflow
always runs and the required Lint/Test/Build Package checks report on
docs-only PRs (previously skipped entirely, blocking merges).
Drop the github.event_name == 'push' gate on the build job so the
required Build Package check also reports on pull requests.
Replace the lingering v0.9 'Breaking Changes' alert with a concise v0.14
'DEPRECATION NOTICE' for SQLite (dual compatibility for ~3 months, then
PostgreSQL-only).
Move all database-specific instructions (SQLite + PostgreSQL) out of the
README into a new canonical docs/database.md covering:
- SQLite zero-config default (DATA_HOME / meshcore.db, WAL/single-host)
- PostgreSQL: DATABASE_* env vars, bundled Docker profile, production
role/database provisioning (mirrors the ipnet-mesh/infrastructure init
script), managed/external Postgres and DATABASE_URL
- Schema-per-instance (search_path) isolation for multiple instances on a
shared cluster
- Pointer to the SQLite->PostgreSQL migration runbook in upgrading.md
Update the README Multi-Instance Deployments and Scaling the API sections
to link to docs/database.md, and add the new doc to the docs list and
project tree. Add a pointer in .env.example and a Postgres note in
AGENTS.md.
Consolidate docs/upgrading.md v0.14: move the env-var/schema/provisioning
reference to docs/database.md (single source of truth) and keep only the
upgrade-time migration runbook and dashboard chart fix.
The packets feature was the only flag gated with a strict `=== true` check
in the SPA JavaScript while every other flag uses `!== false`. Align packets
to the shared pattern across route registration, mobile nav, page titles, the
home nav card, and the messages/advertisements packet-detail link gates.
Dashboard charts (activity, message-activity, node-count) rendered as
flat zeros on Postgres because func.date() returns a str on SQLite but
a datetime.date on Postgres — the dict lookup by string key always
missed. Fixed with a dialect-neutral _date_bucket_key() helper and
pinned the Postgres session timezone to UTC at the engine level.
Also adds dual-backend test infrastructure (TEST_DATABASE_BACKEND env
var), per-worker Postgres databases for pytest-xdist isolation, and
strengthened regression tests asserting non-zero date buckets.
After the Postgres migration, nodes with no last_seen timestamp floated
to the top of the default list because Postgres sorts NULLs first under
ORDER BY ... DESC, whereas SQLite (the previous backend) sorts them last.
Wrap the last_seen ORDER BY with SQLAlchemy nullslast() so NULL last_seen
nodes always sink to the end regardless of database backend or sort
direction. Adds three regression tests covering DESC, ASC, and all-NULL
cases.
renovate[bot]