- compose: add optional postgres service (postgres:17-alpine, profile
'postgres', healthcheck, postgres_data volume); POSTGRES_* derive from
DATABASE_* (single source of truth). DATABASE_* env added to migrate/
collector/api; migrate depends_on postgres with required:false so SQLite
deployments are unaffected.
- alembic/env: resolve the URL via CommonSettings.effective_database_url so
DATABASE_BACKEND=postgres is honoured (previously DATABASE_URL/DATA_HOME
only -> would silently migrate SQLite).
- migrations: normalize_public_key uses STRING_AGG + HAVING COUNT(*) on
Postgres (was SQLite GROUP_CONCAT + alias); raw_packets uses sa.JSON()
not the sqlite dialect type.
- database: fix _to_async_url to map postgresql+psycopg2:// (what the config
assembles) to asyncpg, so API async sessions work on Postgres; resolve the
search_path schema from DATABASE_SCHEMA env when not passed explicitly.
Validated against a live postgres:17: db upgrade builds all 13 tables in the
meshcorehub schema with correct native types (is_observer boolean, decoded
json) and alembic_version stamped; the upsert, JSON/timestamptz round-trip,
and asyncpg async sessions all work. SQLite suite still green (1061 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three related performance fixes for slow (>500ms) API responses.
1. Stop blocking the event loop. Route handlers were declared `async def`
but ran synchronous SQLAlchemy queries (and synchronous Redis calls via
the cache decorator) directly on the event loop, serializing requests.
Convert all handlers to sync `def` so FastAPI runs them in its
threadpool, and make the `@cached` decorator dual-mode (sync wrapper for
sync handlers, async wrapper preserved for a future async/Postgres path).
2. Tune SQLite for concurrency. Enable WAL, busy_timeout and
synchronous=NORMAL on every connection, and size the pool above the
threadpool so handlers don't wait on connections. In-memory SQLite is
guarded (no overflow-pool kwargs).
3. Precompute an indexed `nodes.is_observer` flag. The `observer=true`
filter scanned ~68k+ event rows to find a handful of observers (the
advertisements page calls it with limit=500). Replace the 5-way OR of
subqueries with `WHERE nodes.is_observer = ?`. The collector sets the
flag on first observation (in add_event_observer); the cleanup job
clears it once a node's events are all pruned; a migration adds the
column/index and backfills from the existing union.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Louis King