Commit Graph

2 Commits

Author SHA1 Message Date
Louis King 34b6e6b328 test: cover Postgres migration helper and CLI command
Raise patch coverage on the v0.14.0 Postgres backend:

- test_db_migrate: _is_superuser plus the full migrate_sqlite_to_postgres
  flow (dry-run, copy, non-empty-target refusal, --truncate, missing schema),
  routing create_database_engine to SQLite files so a postgresql:// target
  satisfies the guard while the run executes SQLite -> SQLite in CI.
- test_main: the `db migrate-to-postgres` CLI command via CliRunner
  (success, dry-run, row-count mismatch, ValueError -> ClickException).
- conftest: neutralise dotenv.load_dotenv before collection. Importing the
  CLI entrypoint runs load_dotenv() at import time, which leaked a local .env
  into os.environ and broke unrelated config/redis tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 21:11:02 +01:00
Louis King 9a5c6e9117 Phase 4: add 'db migrate-to-postgres' data migration command
Copies an existing SQLite database into Postgres at the SQLAlchemy Core level,
iterating Base.metadata.sorted_tables (parent-first; excludes alembic_version)
and round-tripping each row through the typed columns so booleans, JSON, and
timestamptz convert correctly with no per-model code.

- streams large tables (stream_results + partitions) in batches
- stamps UTC on naive datetimes for tz-aware columns before insert
- single target transaction (all-or-nothing); refuses a non-empty target
  unless --truncate; --dry-run previews per-table counts
- disables FK triggers via session_replication_role only when the target role
  is a superuser, else relies on parent-first order (--no-replication-role to
  force; managed Postgres). Defaults: source = SQLite under DATA_HOME,
  target = configured DATABASE_* (schema-scoped).
- prints a per-table source->target reconciliation and fails on mismatch

Validated end-to-end against live postgres:17 (nodes/observers/raw_packets/
channels): counts reconcile, dedup preserved, is_observer->boolean,
decoded->json, received_at->timestamptz (UTC). SQLite suite green (1064).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 22:45:29 +01:00