Add actions/delete-package-versions@v5 to both Docker build pipelines to
clean up orphaned untagged manifests left behind by multi-arch rebuilds.
Uses delete-only-untagged-versions so tagged versions (v*, latest, main,
sha-*) are never deleted, with min-versions-to-keep: 10 to protect the
freshly-pushed image's own untagged child/attestation manifests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SPA reaches the backend via the web api_proxy, which forwarded query
params using dict(request.query_params). dict() on Starlette's QueryParams
multidict keeps only the last value of a repeated key, so a multi-valued
filter like ?observed_by=A&observed_by=B was forwarded to the backend as
observed_by=B only. The backend then filtered to B's events, making a
message observed only by A disappear as soon as B was also selected — the
reported "filters act like AND" symptom. This happens independently of the
Redis response cache.
Forward request.query_params.multi_items() (a list of (key, value) tuples)
so all repeated values reach the backend. Add web proxy regression tests
asserting both observed_by values are forwarded, and capture forwarded
params in the MockHttpClient.
This is the primary fix; the earlier cache-key change (multi_items in
sorted_query_string) addressed the same collapse pattern at the cache layer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The API response cache key was built from request.query_params.items(),
which in Starlette keeps only the last value of a repeated key. Repeated
params like observed_by (?observed_by=A&observed_by=B) therefore collapsed
to observed_by=B in the key, colliding with any other filter set sharing the
same last value. The first response cached under that key was then served
for the whole TTL, making observer-filtered Messages/Adverts return stale,
wrong results (e.g. an A-only message vanishing when B was also enabled).
Use multi_items() so all repeated values are included, sorted by the full
(key, value) tuple so order-independent filter sets map to one key. Add
regression tests covering preservation, order-independence, and the
{A,B} vs {B} collision case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The org.opencontainers.image.source label was pointing at
meshcore-dev/meshcore-hub (the upstream MeshCore project's org)
instead of this repo's canonical location at ipnet-mesh/meshcore-hub.
Verify via: docker inspect ghcr.io/ipnet-mesh/meshcore-hub:main
Add concurrency (PR-cancel only), dependency caching, timeouts, and
path filters across all four workflows. Pin opencode action to v1.17.7
and tighten the /oc trigger. Skip MQTT broker rebuild when upstream
SHA is unchanged. Gate sdist/wheel build job to main-only pushes.
Raise patch coverage on the v0.14.0 Postgres backend:
- test_db_migrate: _is_superuser plus the full migrate_sqlite_to_postgres
flow (dry-run, copy, non-empty-target refusal, --truncate, missing schema),
routing create_database_engine to SQLite files so a postgresql:// target
satisfies the guard while the run executes SQLite -> SQLite in CI.
- test_main: the `db migrate-to-postgres` CLI command via CliRunner
(success, dry-run, row-count mismatch, ValueError -> ClickException).
- conftest: neutralise dotenv.load_dotenv before collection. Importing the
CLI entrypoint runs load_dotenv() at import time, which leaked a local .env
into os.environ and broke unrelated config/redis tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Messages containing runs of newlines were preserved by the pre-wrap
styling on the messages view, stretching table rows and cards. Collapse
any run of newlines (and surrounding whitespace) into a single space at
display time, leaving stored text untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FastAPI 0.137.0 regressed include_router: endpoints registered via
app.include_router no longer appear in app.routes (they still serve, but
introspection breaks). tests/test_api/test_app_factory.py asserts on
app.routes, so CI (which pip-installs the latest fastapi) failed on the
/metrics route check while local pinned 0.136.3 passed. Constrain fastapi
to <0.137.0 until a fixed release is available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The web_app fixture already neutralizes NETWORK_ANNOUNCEMENT so a developer's
local .env does not leak into tests. Do the same for the two new system
settings, fixing test_system_banner_absent_when_none when SYSTEM_ANNOUNCEMENT
is set in the environment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the #flash-banner flex centering rule to #system-banner so it matches
the network announcement layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire the two new web settings into the web service env block, matching the
NETWORK_ANNOUNCEMENT passthrough. SYSTEM_MAINTENANCE defaults to false.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add two operator-controlled, startup-time settings:
- SYSTEM_ANNOUNCEMENT: non-dismissable Markdown banner rendered above the
network announcement on every page (navbar -> system -> network).
- SYSTEM_MAINTENANCE: when enabled, forces all feature flags off so the nav
collapses to Home, hides the profile menu, and the SPA renders a maintenance
page for every route. The maintenance page makes no API calls, so the API
and database can be offline while the web component keeps running.
CLI exposes --system-announcement and tri-state --system-maintenance; the bool
falls back to pydantic settings to parse SYSTEM_MAINTENANCE reliably from env.
Adds i18n strings (en/nl), tests, and docs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the multi-select Observer dropdown buried in the filter panel with
a row of clickable observer badges rendered between the filter panel and the
data list (and below the Sorting dropdown on mobile).
- Selection is stored in localStorage (shared across Adverts and Messages)
as the disabled set, so new observers default to enabled.
- Badge style reflects enabled (filled) vs disabled (muted) state; the last
enabled observer cannot be toggled off.
- Observer filter is sourced from localStorage instead of the URL query;
a two-phase fetch resolves the enabled include-list (the API filters by
inclusion only) before fetching data, with no flash of unfiltered results.
- Toggling re-scopes data and resets to page 1.
- Add enable/disable tooltip strings (en/nl) and the previously-missing
Dutch observer label.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PostgreSQL support ships in v0.14.0; v0.13.0 covers the packet-handling
enhancements. Reparent the "Optional PostgreSQL Backend" notes accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The column was declared String(8) but stores 16-char hex signatures (and can
hold the up-to-32-char packet_hash fallback from the LetsMesh normalizer).
SQLite never enforced the length, so the undersized definition went unnoticed
until db migrate-to-postgres rejected the data with
StringDataRightTruncation (varchar(8)). Widen the model column and add an
Alembic migration (batch_alter_table, so it applies on both SQLite and
Postgres). Verified end-to-end migrating a 1.2GB live SQLite DB into Postgres
(all 12 tables reconcile).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Settings classes use env_file=".env" for deployments, which also caused tests
to pick up a developer's local .env (e.g. DATABASE_BACKEND=postgres without
DATABASE_HOST), failing unrelated API/CLI tests. Add an autouse fixture that
sets env_file=None on CommonSettings and every subclass, so tests depend only
on defaults and explicit monkeypatch.setenv overrides.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a Database Backend section to the README (config vars, compose profile,
schema-per-instance) and an "Optional PostgreSQL Backend" section to
docs/upgrading.md covering enablement, search_path isolation, role/db
provisioning, and the SQLite -> Postgres data-migration runbook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The runtime image installed the package without extras, so psycopg2/asyncpg
were missing and the migrate/collector/api services failed with
ModuleNotFoundError when DATABASE_BACKEND=postgres. Install ".[postgres]" so
the one image serves both SQLite and Postgres.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copies an existing SQLite database into Postgres at the SQLAlchemy Core level,
iterating Base.metadata.sorted_tables (parent-first; excludes alembic_version)
and round-tripping each row through the typed columns so booleans, JSON, and
timestamptz convert correctly with no per-model code.
- streams large tables (stream_results + partitions) in batches
- stamps UTC on naive datetimes for tz-aware columns before insert
- single target transaction (all-or-nothing); refuses a non-empty target
unless --truncate; --dry-run previews per-table counts
- disables FK triggers via session_replication_role only when the target role
is a superuser, else relies on parent-first order (--no-replication-role to
force; managed Postgres). Defaults: source = SQLite under DATA_HOME,
target = configured DATABASE_* (schema-scoped).
- prints a per-table source->target reconciliation and fails on mismatch
Validated end-to-end against live postgres:17 (nodes/observers/raw_packets/
channels): counts reconcile, dedup preserved, is_observer->boolean,
decoded->json, received_at->timestamptz (UTC). SQLite suite green (1064).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- compose: add optional postgres service (postgres:17-alpine, profile
'postgres', healthcheck, postgres_data volume); POSTGRES_* derive from
DATABASE_* (single source of truth). DATABASE_* env added to migrate/
collector/api; migrate depends_on postgres with required:false so SQLite
deployments are unaffected.
- alembic/env: resolve the URL via CommonSettings.effective_database_url so
DATABASE_BACKEND=postgres is honoured (previously DATABASE_URL/DATA_HOME
only -> would silently migrate SQLite).
- migrations: normalize_public_key uses STRING_AGG + HAVING COUNT(*) on
Postgres (was SQLite GROUP_CONCAT + alias); raw_packets uses sa.JSON()
not the sqlite dialect type.
- database: fix _to_async_url to map postgresql+psycopg2:// (what the config
assembles) to asyncpg, so API async sessions work on Postgres; resolve the
search_path schema from DATABASE_SCHEMA env when not passed explicitly.
Validated against a live postgres:17: db upgrade builds all 13 tables in the
meshcorehub schema with correct native types (is_observer boolean, decoded
json) and alembic_version stamped; the upsert, JSON/timestamptz round-trip,
and asyncpg async sessions all work. SQLite suite still green (1061 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- config: add DatabaseBackend enum + DATABASE_* component vars on
CommonSettings (host/port/name/schema/user/password, defaults
'meshcorehub'); single effective_database_url resolver with precedence
DATABASE_URL > postgres (assembled, fail-fast on missing vars) > SQLite
default. effective_database_schema returns the schema only for Postgres.
Collapses the duplicated resolver/field out of Collector/API settings.
- database: create_database_engine/DatabaseManager accept a schema arg and
scope Postgres connections via search_path (psycopg2 -c options; asyncpg
server_settings). No-op for SQLite.
- alembic/env: migrate into the instance schema (version_table_schema +
CREATE SCHEMA + search_path) for Postgres; fix online render_as_batch to
be SQLite-only (matching the offline path).
- .env.example: document the DATABASE_* block.
SQLite behaviour unchanged: default no-env path resolves to the same URL
(new regression tests), full suite green (1051 passed), fresh db upgrade OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dialect-neutral fixes that keep SQLite behaviour unchanged:
- event_observer: choose INSERT construct by bind dialect so the
on_conflict upsert works on both SQLite and Postgres
- database: map postgresql:// -> postgresql+asyncpg:// for async sessions
(was sqlite-only), via a _to_async_url helper
- models: import JSON from sqlalchemy (generic) not the sqlite dialect
- alembic/env: render_as_batch only for SQLite (its ALTER TABLE workaround)
Full SQLite test suite green (1045 passed) and fresh db upgrade builds all
tables; black/flake8/mypy clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the DATABASE_BACKEND explicit switch, schema-per-instance isolation
via search_path for shared-cluster prod/stg, the no-admin-credentials
provisioning decision, the migrate-to-postgres command internals, the
operator runbook, and the phased implementation order with SQLite gates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Louis King