diff --git a/docs/plans/20260616-2023-fix-postgres-charts-flatline/plan.md b/docs/plans/20260616-2023-fix-postgres-charts-flatline/plan.md new file mode 100644 index 0000000..08b4fe5 --- /dev/null +++ b/docs/plans/20260616-2023-fix-postgres-charts-flatline/plan.md @@ -0,0 +1,267 @@ +# Fix flatlined dashboard charts on Postgres + +## Summary + +After the Postgres migration (`feat/postgres-support`, merged in `1ba3e17`), the activity +charts on the home page and the message-activity and node-count charts on the network page +render as flat zeros. Switching `DATABASE_BACKEND` back to `sqlite` makes them work again, +which pinpoints the regression to a dialect-specific code path rather than missing data. + +The three affected endpoints — `GET /api/v1/dashboard/activity`, +`GET /api/v1/dashboard/message-activity`, and `GET /api/v1/dashboard/node-count` — all group +rows with `func.date()` and then look the result up by a `"%Y-%m-%d"` +string. On SQLite `func.date()` returns a Python `str`; on Postgres it returns a +`datetime.date` object. The dict built from the result rows is therefore keyed by a different +type than the string used for lookup, so every `.get(date_str, 0)` returns the `0` default and +the chart flatlines — even though the underlying query returns the correct rows. + +The fix is a **single, dialect-neutral code path** that runs unchanged on both SQLite and +Postgres: keep `func.date()` uniformly and coerce the returned key to a canonical string in +Python, then guarantee the Postgres session is UTC so the day boundary matches SQLite's UTC-text +truncation. No `if dialect == ...` branches in the query layer — both backends execute the same +SQL and the same normalization. This is paired with the regression coverage the Postgres +migration plan promised but never delivered: a SQLite + Postgres test matrix that asserts the +dashboard endpoints return identical results on both backends. + +## Background & Motivation + +The Postgres migration (`docs/plans/20260613-2111-postgres-migration/plan.md`) shipped in +`1ba3e17` and was followed by `a554e09` (revert to Postgres 17) and `8bf4536` (node-list +NULLs-last fix). The migration plan was explicit that **Phase 3, Gate 3** would "wire a SQLite ++ Postgres test matrix here so both run going forward." That matrix was never implemented: +`tests/test_api/conftest.py:56` hard-codes `sqlite:///{test_db_path}`, and every dashboard +test (`tests/test_api/test_dashboard.py`) runs only against SQLite. On SQLite, +`func.date()` returns the expected `"%Y-%m-%d"` string, so the dict lookups succeed and all +tests pass — the bug is invisible to the suite. + +The dashboard route file (`src/meshcore_hub/api/routes/dashboard.py`) uses `func.date()` in +three places: + +| Line | Endpoint | Chart | +|------|----------|-------| +| 296 | `get_activity` | Home page advert activity | +| 356 | `get_message_activity`| Network page message activity | +| 428 | `get_node_count_history` | Network page cumulative node count (also `new_by_date` at L435) | + +All three follow the identical pattern: + +```python +date_expr = func.date(Advertisement.received_at) # SQLite: str, PG: date obj +... +counts_by_date = {row.date: row.count for row in results} # dict keyed by whatever DB returns +... +date_str = date.strftime("%Y-%m-%d") # always str +count = counts_by_date.get(date_str, 0) # str lookup vs. date-obj key -> always 0 on PG +``` + +The query itself is valid on both dialects (Postgres resolves `date(col)` to the timestamp→date +cast function), so there is no error — the rows come back, the cache layer (`@cached(...)` on +each endpoint) happily caches the all-zero response, and the chart renders zeros. This is why +the symptom is "flatlined at 0" rather than a 500. + +A secondary, latent concern: on Postgres `func.date()` truncates to the *session* +timezone's date. The collector writes UTC (`utc_now()`), so if the connection's `timezone` +setting ever drifts from UTC the bucket boundaries would shift. This plan addresses it as part +of the fix by making the bucket UTC-explicit. + +## Goals +- Restore correct chart data on Postgres for `/dashboard/activity`, + `/dashboard/message-activity`, and `/dashboard/node-count` (home + network pages). +- **Make both backends behave identically**: the dashboard endpoints must return the same + buckets for the same seed data on SQLite and Postgres, driven by one shared code path with + no per-dialect branches in the query layer. +- Add regression coverage by parameterizing the full API test suite to support **both** SQLite + and Postgres backends — closing the gap the migration plan's Gate 3 left open — so the two + backends can be tested manually against the same suite. Postgres testing is run locally, not + in CI. +- Confirm the cached layer (`@cached`) does not serve stale all-zero responses after the fix + (verified: the default dashboard TTL is 30 seconds, so stale entries expire before anyone + notices post-deploy). + +## Non-Goals +- Redesigning the dashboard endpoints or their response schemas. +- Introducing a per-dialect query branch (e.g. `func.to_char(...)` on Postgres vs + `func.date()` on SQLite) — the fix must be one code path on both backends. +- Changing what timezone the *collector* writes (it already writes UTC). +- Revisiting the multi-instance `search_path` / schema work from the migration plan. +- Migrating any other SQLite-specific call sites beyond what the charts need (a separate audit + can follow; `event_observer.py` already has its dialect-aware upsert). +- Frontend changes — the SPA (`charts.js`) consumes the same JSON shape and needs no edits. +- Adding a CI Postgres test matrix job — Postgres backend testing is manual/local only. + +## Requirements + +### Functional Requirements +- `/dashboard/activity`, `/dashboard/message-activity`, and `/dashboard/node-count` return the + same non-zero day buckets on Postgres as they do on SQLite for identical data. +- **Cross-backend equivalence:** for the same seed rows, the three endpoints must return + byte-for-byte identical JSON on SQLite and Postgres. The fix is unified, not a per-backend + special case that happens to agree. +- Behavior on SQLite is unchanged (no regression for the default zero-config backend). +- No explicit cache-invalidation step is required for the default 30-second dashboard TTL; + operators who have configured a substantially longer `REDIS_CACHE_TTL_DASHBOARD` should + be documented in the upgrade notes as needing to wait one TTL or flush the three + dashboard key prefixes. + +### Technical Requirements +- **One unified code path, not two.** The date-bucketing logic must be identical on SQLite and + Postgres: a single `func.date(col)` SQL construct and a single Python-side key coercion. No + `dialect.name == "postgresql"` / `dialect.name == "sqlite"` branch in `dashboard.py` (the + `event_observer.py:144-152` upsert branch is the *only* dialect fork that should exist, and + only because there is no truly portable equivalent). +- Date keys must be normalized to a canonical `"%Y-%m-%d"` string **before** being used as dict + keys, independent of the DB driver's return type. The normalization must handle both `str` + (SQLite) and `datetime.date`/`datetime.datetime` (Postgres) inputs. +- **Postgres session timezone must be UTC** so `func.date()` truncates to the UTC + day boundary — matching SQLite, which stores UTC text and truncates to the date portion. Set + this once per connection at the engine level (psycopg2 `connect_args["options"]` + + `-ctimezone=UTC` for the sync engine, `server_settings={"timezone": "UTC"}` for asyncpg), + not per-query. This is the single infra-level guarantee that lets the query layer stay + dialect-agnostic. +- The fix must not change the SQL semantics on SQLite (string-grouping via `func.date()` stays). +- A regression test must assert at least one bucket has `count >= 1` for each endpoint and must + run against **both** SQLite and Postgres (parameterized fixture / test matrix), asserting the + two backends return the same buckets for the same seed rows. +- Tests must clear or bypass the Redis cache layer so they assert the query, not the cache. + +## Implementation Plan + +### Phase 1: Confirm root cause against a live Postgres +- Spin up a throwaway Postgres (`postgres:17-alpine`, port `55432`), `db upgrade`, copy a slice + of recent `advertisements`/`messages`/`nodes` from `data/collector/meshcore.db` via the + existing `db migrate-to-postgres` command (or a trimmed manual insert). +- Hit the three endpoints and confirm zeros; run the raw `func.date()` `GROUP BY` query by + hand and observe that rows come back with `date` objects. +- This step is verification only — it produces no code. + +### Phase 2: Unified date-key fix (single code path for both backends) +- Add a small helper in `src/meshcore_hub/api/routes/dashboard.py` (module-private, e.g. + `_date_bucket_key(value) -> str`) that coerces any DB-returned value to a + `"%Y-%m-%d"` string: pass through `str` unchanged, call `.strftime("%Y-%m-%d")` on + `date`/`datetime`, return `None` unchanged (shouldn't happen for the types the DB + drivers return, but the companion `None` key won't collide with any `date_str` lookup), + and leave everything else alone. (One-line isinstance ladder.) +- Apply it when building `counts_by_date` / `new_by_date` in all three endpoints + (`dashboard.py:313`, `:377`, `:435`): `{_date_bucket_key(row.date): row.count ...}`. + At L435 (the `node-count` `per_day_query`), the existing code uses `row[0]: row[1]` index + access; switch it to `row.date: row.count` (the query already labels the columns + `.label("date")` / `.label("count")`) so all three endpoints follow the same named-access + pattern. The lookup side (`counts_by_date.get(date_str, 0)` / `new_by_date.get(date_str, 0)`) + is already string-based and needs no change. +- **Keep `func.date(col)` uniformly on both dialects** — do not introduce a Postgres + `func.to_char(...)` branch. The two backends emit slightly different SQL for `func.date()` + (SQLite's scalar function vs. Postgres' timestamp→date cast), but SQLAlchemy's `func.date()` + compiles correctly on both, and the result-row normalization above makes the Python-side key + identical either way. This is the "both backends work the same" guarantee: one query + construct, one coercion, no branch. +- **Pin the Postgres session timezone to UTC** at engine creation in + `src/meshcore_hub/common/database.py` (alongside the existing SQLite `PRAGMA` block, which is + already dialect-guarded). For the sync engine (psycopg2): append `-ctimezone=UTC` to the + `connect_args["options"]` string (alongside the existing `search_path` `-c` flag; the + timezone flag must be set unconditionally for all Postgres connections, not gated on + `resolved_schema` — a test Postgres engine without a custom schema still needs UTC). For the + async engine (asyncpg): add `"timezone": "UTC"` to the `server_settings` dict (the same + mechanism already used for `search_path` at `database.py:201`; similarly set it + unconditionally, not only when a schema is present). SQLite needs nothing — it has + no session timezone and already stores UTC text. This single connection-level guarantee makes + `func.date()` truncate on the UTC day boundary exactly as SQLite does, so the + two backends bucket identically. +- No schema migration, no model change, no Alembic revision. + +> **Why not a dialect branch (rejected):** `func.to_char(col.op('AT TIME ZONE')('UTC'), +> 'YYYY-MM-DD')` on Postgres would return a string directly and sidestep the coercion, but it +> requires a `dialect.name` fork in `dashboard.py` and diverges the two code paths. The +> requirement is that both backends *work the same*, so the unified `func.date()` + Python +> coercion + UTC-session approach is preferred; the helper is kept as a defensive guard for any +> future driver that returns a non-string. + +### Phase 3: Regression tests + the missing SQLite/Postgres matrix +- Parameterize the full API test suite (`tests/test_api/`) to run against both SQLite and + Postgres. Scope: every API endpoint, not just the dashboard — any endpoint calling + `func.date()` or similar SQLAlchemy constructs could have a latent dialect type mismatch. +- Factor the database engine fixture in `tests/test_api/conftest.py` so the backend is + selected by a fixture param (or an env var such as `TEST_DATABASE_BACKEND`), defaulting to + SQLite so the local `make test` loop is unchanged. When Postgres is requested, build the URL + from the same `DATABASE_*` env vars used in production (or a dedicated `TEST_POSTGRES_URL`). + Postgres runs are manual (spin up a local `postgres:17-alpine`, set the env var, run pytest); + no CI job is added. +- Ensure the dashboard-specific tests assert the two backends return the same buckets for the + same seed rows (cross-backend equivalence check). +- The test suite already bypasses Redis: the `@cached` decorator checks + `request.app.state.redis_cache` which is `None` in the test `create_app()` (no Redis + configured), falling through to the raw handler. No action needed. + +### Phase 4: Live verification + +- Bring the stack up on the `postgres` profile against a real dataset and confirm the three + charts render non-zero buckets matching the SQLite baseline. +- No cache-invalidation step is required: the default `REDIS_CACHE_TTL_DASHBOARD` is 30 + seconds (`config.py:314` → `app.py:93`), so stale all-zero cached responses expire before + anyone views the dashboard post-deploy. If the operator has set a substantially longer TTL, + document the one-TTL wait in the upgrade notes. +- Spot-check a known-busy day against a raw `SELECT date(...), count(*) ...` to confirm the + rendered value matches. + +## Open Questions + +- **UTC-session application:** set `timezone=UTC` on all engines — API, collector, migrate, + and the `db migrate-to-postgres` one-shots — via the shared `create_database_engine()` and + `DatabaseManager.__init__()` so there is one place that could be wrong. This gives the + migration copy a consistent day boundary with the runtime queries. Resolved: apply + everywhere. + +## References +- `docs/plans/20260613-2111-postgres-migration/plan.md` — the migration that introduced the + regression; its Phase 3 Gate 3 promised the SQLite+Postgres test matrix this plan delivers. +- `src/meshcore_hub/api/routes/dashboard.py:296,356,428,435` — the `func.date()` call sites and + dict-key mismatch. +- `tests/test_api/conftest.py:56` — SQLite-only test engine that hid the bug. +- `tests/test_api/test_dashboard.py` — existing dashboard tests (all SQLite-only today). +- `src/meshcore_hub/common/models/event_observer.py:144-152` — the existing dialect-aware + upsert pattern; noted as the *only* dialect fork that should exist in the codebase (no truly + portable SQL equivalent), and explicitly not mirrored by this fix. +- Git: `1ba3e17` (merge of `feat/postgres-support`), `a554e09` (revert to PG 17), + `34410de` (Postgres 18 → 17 revert merge). + +## Review + +**Status**: Approved + +**Reviewed**: 2026-06-16 + +### Resolutions + +- **Test matrix scope** — Parameterize the **full API test suite** to support both SQLite and + Postgres backends, not just the dashboard tests. Any endpoint using SQLAlchemy constructs could + have a latent dialect type mismatch; surfacing it all at once is cheaper than chasing + one-by-one regression bugs. Postgres testing is run manually (local container), not in CI. + +- **Cache invalidation** — No explicit cache flush needed. The default dashboard TTL is 30 + seconds (`config.py:314`, `app.py:93`); stale all-zero cached responses expire before a + user loads the dashboard post-deploy. For operators with a substantially longer + `REDIS_CACHE_TTL_DASHBOARD`, document the one-TTL wait in the upgrade notes. + +- **UTC-session application** — Apply `timezone=UTC` on **all** engines (API, collector, + migrate, `db migrate-to-postgres`) via the shared `create_database_engine()` and + `DatabaseManager.__init__()` helpers, so there is a single point of control. + +- **_date_bucket_key type coverage** — Handle `None` by returning it unchanged (no + collision with string lookups). Clarified in the plan. + +- **asyncpg timezone mechanism** — Use `server_settings={"timezone": "UTC"}` in asyncpg's + `connect_args`, matching the existing `search_path` pattern at `database.py:201`, rather + than a separate connect-event listener. + +- **Postgres timezone not gated on schema** — The `-ctimezone=UTC` flag must be set + unconditionally for all Postgres connections in `create_database_engine()`, not gated on + `resolved_schema` being non-empty (a Postgres test engine without a custom schema still + needs UTC). + +- **node-count endpoint row access** — L435 (`dashboard.py`) uses `row[0]` index access + while L313/L377 use `row.date` named access. The fix will switch L435 to named access + (`row.date`, `row.count`) for consistency across all three endpoints. + +### Remaining Action Items + +- For deployments with a non-default (long) `REDIS_CACHE_TTL_DASHBOARD`, add a one-TTL + wait note to `docs/upgrading.md`. diff --git a/docs/plans/20260616-2023-fix-postgres-charts-flatline/tasks.md b/docs/plans/20260616-2023-fix-postgres-charts-flatline/tasks.md new file mode 100644 index 0000000..427f3fa --- /dev/null +++ b/docs/plans/20260616-2023-fix-postgres-charts-flatline/tasks.md @@ -0,0 +1,106 @@ +# Tasks: Fix flatlined dashboard charts on Postgres + +> Generated from `plan.md` on 2026-06-16 + +## 1. Postgres UTC Session Configuration + +- [x] **1.1** Add unconditional `-ctimezone=UTC` to the sync engine in `create_database_engine()` (`database.py:65-94`) + - [x] In the Postgres branch (after the SQLite `check_same_thread` guard, near L76), ensure `connect_args["options"]` always includes `-ctimezone=UTC` — even when `resolved_schema` is empty + - [x] When `resolved_schema` is set, combine both flags: `-csearch_path= -ctimezone=UTC` + - [x] When `resolved_schema` is empty/None, set `connect_args["options"] = "-ctimezone=UTC"` alone + - [x] Gate the entire block on `database_url` being Postgres (not SQLite) — mirror the existing `database_url.startswith("sqlite")` pattern +- [x] **1.2** Add unconditional `"timezone": "UTC"` to the async engine in `DatabaseManager._ensure_async_engine()` (`database.py:188-204`) + - [x] Ensure `async_connect_args["server_settings"]` always includes `"timezone": "UTC"` for Postgres + - [x] When `self._schema` is set, merge into one `server_settings` dict: `{"search_path": schema, "timezone": "UTC"}` + - [x] When `self._schema` is None (Postgres without custom schema), still create `server_settings = {"timezone": "UTC"}` + - [x] Only apply for Postgres URLs, not SQLite (SQLite async engine has no `server_settings`) +- [x] **1.3** Add unit tests verifying UTC timezone is configured on Postgres engines + - [x] Test that `create_database_engine()` with a `postgresql://` URL sets `connect_args["options"]` containing `-ctimezone=UTC` + - [x] Test that `create_database_engine()` with a Postgres URL + schema sets both `-csearch_path=` and `-ctimezone=UTC` in options + - [x] Test that `create_database_engine()` with a `sqlite://` URL does NOT set timezone options + - [x] Test that `DatabaseManager` async engine has `server_settings` with `"timezone": "UTC"` for Postgres URLs + +## 2. Unified Date-Key Normalization + +- [x] **2.1** Add `_date_bucket_key(value) -> str | None` helper in `src/meshcore_hub/api/routes/dashboard.py` + - [x] Return `value` unchanged if it's already a `str` + - [x] Return `value.strftime("%Y-%m-%d")` if it's a `datetime.date` or `datetime.datetime` + - [x] Return `None` unchanged if `value is None` (won't collide with string lookups) + - [x] Use a single `isinstance` ladder — no dialect check + - [x] Add module-level `from datetime import date as date_type, datetime as datetime_type` imports if needed (or import `datetime` already present) +- [x] **2.2** Apply `_date_bucket_key` to `get_activity` endpoint (`dashboard.py:313`) + - [x] Change `{row.date: row.count for row in results}` to `{_date_bucket_key(row.date): row.count for row in results}` + - [x] Update the comment on L312 from "date is already a string" to reflect the normalization +- [x] **2.3** Apply `_date_bucket_key` to `get_message_activity` endpoint (`dashboard.py:377`) + - [x] Change `{row.date: row.count for row in results}` to `{_date_bucket_key(row.date): row.count for row in results}` +- [x] **2.4** Apply `_date_bucket_key` to `get_node_count_history` endpoint (`dashboard.py:435-436`) + - [x] Switch from index access `row[0]: row[1]` to named access: `{_date_bucket_key(row.date): row.count for row in session.execute(per_day_query).all()}` + - [x] The query already labels columns `.label("date")` and `.label("count")` (L430), so named access works +- [x] **2.5** Add unit tests for `_date_bucket_key` helper + - [x] Test with a `str` input (e.g. `"2026-06-15"`) — returns unchanged + - [x] Test with a `datetime.date` input — returns `"%Y-%m-%d"` string + - [x] Test with a `datetime.datetime` input — returns `"%Y-%m-%d"` string + - [x] Test with `None` — returns `None` + - [x] Test that `datetime.date(2026, 1, 5)` produces `"2026-01-05"` (zero-padded) + +## 3. Dual-Backend Test Infrastructure + +- [x] **3.1** Add backend-selection mechanism to `tests/test_api/conftest.py` + - [x] Add a `TEST_DATABASE_BACKEND` env var check (or `TEST_POSTGRES_URL`), defaulting to `sqlite` so local `make test` is unchanged + - [x] Add a session-scoped `db_backend` fixture that reads the env var and returns `"sqlite"` or `"postgres"` + - [x] Skip Postgres tests if `TEST_DATABASE_BACKEND=postgres` is requested but no Postgres is reachable (use `pytest.fixture` with `pytest.skip()`) +- [x] **3.2** Refactor `api_db_engine` fixture to support both backends (`conftest.py:48-71`) + - [x] When backend is SQLite: keep existing behavior (temp file, `create_engine`, SQLite pragma) + - [x] When backend is Postgres: build URL from `TEST_POSTGRES_URL` (or `DATABASE_URL`), use `create_engine` with the same `connect_args` as production (including `-ctimezone=UTC` from `create_database_engine`) + - [x] Call `Base.metadata.create_all(engine)` in both cases + - [x] Ensure `Base.metadata.drop_all(engine)` runs on teardown for both backends +- [x] **3.3** Update `mock_db_manager` and app fixtures to use the parameterized engine + - [x] `mock_db_manager` (`conftest.py:110-132`) — bind `sessionmaker` to whichever engine `api_db_engine` yields + - [x] `app_no_auth` (`conftest.py:170-186`) — pass the correct `db_url` (SQLite file path or Postgres URL) to `create_app()` + - [x] `app_with_auth` (`conftest.py:189-199`) — same update + - [x] `_wire_overrides` (`conftest.py:148-167`) — bind `sessionmaker` to parameterized engine +- [x] **3.4** Ensure `_truncate_all` works on Postgres (`conftest.py:74-78`) + - [x] Verify `table.delete()` in reversed FK order works on Postgres (it should — same SQLAlchemy construct) + - [x] If Postgres FK constraints cause issues, consider `TRUNCATE ... CASCADE` as an alternative, but prefer the current approach for backend neutrality +- [x] **3.5** Verify the full existing API test suite passes unchanged on SQLite after refactoring + - [x] Run `pytest --no-cov tests/test_api/` and confirm no regressions from the fixture changes + - [x] Confirm `make test` still works identically with no env var set + +## 4. Regression & Cross-Backend Tests + +- [x] **4.1** Update `tests/test_api/test_dashboard.py` to assert non-zero buckets + - [x] For `get_activity`: seed advertisements with known UTC dates within the window, assert at least one returned `data[].count >= 1` + - [x] For `get_message_activity`: seed messages with known UTC dates, assert at least one `count >= 1` + - [x] For `get_node_count`: seed nodes with `created_at` within the window, assert cumulative count increases (non-zero delta on seeded days) +- [x] **4.2** Add deterministic-date seed fixtures for cross-backend tests + - [x] Create fixtures that insert rows at explicit UTC timestamps (e.g. `datetime(2026, 6, 10, tzinfo=timezone.utc)`) rather than `datetime.now(timezone.utc)` — so both backends see the same bucket boundaries + - [x] Ensure seeded timestamps fall within the endpoint's default day window (use `freezegun` or monkeypatch `utc_now` if needed, or seed dates relative to the current date) +- [x] **4.3** Add cross-backend equivalence test for all three dashboard endpoints + - [x] Create a test that runs against both SQLite and Postgres (via the parameterized fixture) + - [x] Seed identical data on the active backend + - [x] Assert the JSON response from each endpoint has the same `data` array (same dates, same counts) + - [x] This test is the regression guard: if it passes on Postgres, the bug is fixed + +## 5. Documentation + +- [x] **5.1** Add upgrade note to `docs/upgrading.md` for long-TTL operators + - [x] Document that the dashboard chart fix takes effect after one `REDIS_CACHE_TTL_DASHBOARD` period (default 30s) + - [x] For operators with a substantially longer TTL, advise waiting one TTL period or flushing the three dashboard cache key prefixes (`dashboard:activity`, `dashboard:message-activity`, `dashboard:node-count`) + +## 6. Verification + +- [x] **6.1** Run linting and formatting checks + - [x] `pre-commit run --all-files` +- [x] **6.2** Run the full test suite on SQLite + - [x] `pytest -nauto --no-cov` — confirm no regressions (1106 passed, 22 skipped) +- [x] **6.3** Run the full API test suite on Postgres (local, via throwaway container) + - [x] Spin up `postgres:17-alpine` on port 55432 + - [x] `TEST_DATABASE_BACKEND=postgres TEST_POSTGRES_URL=postgresql+psycopg2://postgres:postgres@localhost:55432/test pytest -nauto --no-cov tests/test_api/` + - [x] Confirm all tests pass, especially the new dashboard regression tests (458 passed) +- [ ] **6.4** Live verification against the compose stack + - [ ] Bring the stack up on the Postgres profile with real data (or migrated SQLite data) + - [ ] Hit `GET /api/v1/dashboard/activity`, `/dashboard/message-activity`, `/dashboard/node-count` — confirm non-zero buckets + - [ ] Compare against the same endpoints on SQLite — confirm identical results + - [ ] Spot-check a known-busy day against a raw `SELECT date(...), count(*) ...` query +- [x] **6.5** Verify no `dialect.name` branch was introduced in `dashboard.py` + - [x] Grep `dashboard.py` for `dialect` — confirm zero hits (the fix is dialect-neutral by construction) diff --git a/docs/upgrading.md b/docs/upgrading.md index 08d7154..49f5b32 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -73,6 +73,21 @@ Downtime is required while writers are stopped; the source SQLite file is never > **Managed Postgres / non-superuser roles:** the migration disables foreign-key triggers during the copy via `session_replication_role = replica`, which requires a superuser. When the target role is not a superuser (typical for managed Postgres), the command automatically falls back to copying in parent-first order instead. Pass `--no-replication-role` to force the fallback explicitly. +### Dashboard Chart Fix (Postgres) + +After enabling Postgres, the dashboard charts (activity, message-activity, node-count) may render as flat zeros. This is a known issue caused by a dialect mismatch in the date-bucketing query — `func.date()` returns a `str` on SQLite but a `datetime.date` on Postgres, causing the dict lookup to miss. A fix normalizes the key to a canonical `"%Y-%m-%d"` string and pins the Postgres session timezone to UTC. + +The fix takes effect **after one `REDIS_CACHE_TTL_DASHBOARD` period** (default 30 seconds) — stale all-zero cached responses expire automatically. For operators who have configured a substantially longer TTL, either wait one TTL period or flush the three dashboard cache key prefixes: + +```bash +redis-cli -h DEL \ + "$(echo -n 'hub:dashboard:activity*' | xargs redis-cli KEYS)" \ + "$(echo -n 'hub:dashboard:message-activity*' | xargs redis-cli KEYS)" \ + "$(echo -n 'hub:dashboard:node-count*' | xargs redis-cli KEYS)" +``` + +No database migration or configuration change is required — the fix is automatic. + ## v0.13.0 ### Raw Packets (capture, browse, and search wire packets) diff --git a/pyproject.toml b/pyproject.toml index 6a24fb2..2cbf274 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -64,6 +64,10 @@ dev = [ "beautifulsoup4>=4.12.0", "types-paho-mqtt>=1.6.0", "types-PyYAML>=6.0.0", + # Postgres drivers needed to run the dual-backend test suite + # (TEST_DATABASE_BACKEND=postgres) locally. + "asyncpg>=0.28.0", + "psycopg2-binary>=2.9.0", ] postgres = [ "asyncpg>=0.28.0", diff --git a/src/meshcore_hub/api/routes/dashboard.py b/src/meshcore_hub/api/routes/dashboard.py index 4aea29d..2f26e56 100644 --- a/src/meshcore_hub/api/routes/dashboard.py +++ b/src/meshcore_hub/api/routes/dashboard.py @@ -1,6 +1,6 @@ """Dashboard API routes.""" -from datetime import datetime, timedelta, timezone +from datetime import date, datetime, timedelta, timezone from fastapi import APIRouter, Request from sqlalchemy import and_, case, func, or_, select @@ -60,6 +60,21 @@ def _flood_only_filter( ) +def _date_bucket_key(value: str | date | None) -> str | None: + """Coerce a DB-returned date bucket key to a canonical ``%Y-%m-%d`` string. + + SQLite's ``func.date()`` returns a ``str``; Postgres returns a + ``datetime.date``. This normalizes both to the same string key so dict + lookups by ``"%Y-%m-%d"`` succeed on either backend. ``None`` passes + through unchanged. + """ + if isinstance(value, str): + return value + if isinstance(value, date): + return value.strftime("%Y-%m-%d") + return value + + @router.get("/stats", response_model=DashboardStats) @cached( "dashboard/stats", @@ -309,8 +324,9 @@ def get_activity( results = session.execute(query).all() - # Build a dict of date -> count from results (date is already a string) - counts_by_date = {row.date: row.count for row in results} + # Build a dict of date -> count, normalizing the key to a string so it + # works on both SQLite (func.date() returns str) and Postgres (returns date). + counts_by_date = {_date_bucket_key(row.date): row.count for row in results} # Generate all dates in the range, filling in zeros for missing days data = [] @@ -374,7 +390,7 @@ def get_message_activity( ) results = session.execute(query).all() - counts_by_date = {row.date: row.count for row in results} + counts_by_date = {_date_bucket_key(row.date): row.count for row in results} # Generate all dates in the range, filling in zeros for missing days data = [] @@ -432,8 +448,9 @@ def get_node_count_history( .where(Node.created_at < end_date) .group_by(date_expr) ) - new_by_date: dict[str, int] = { - row[0]: row[1] for row in session.execute(per_day_query).all() + new_by_date = { + _date_bucket_key(row.date): row._mapping["count"] + for row in session.execute(per_day_query).all() } data = [] diff --git a/src/meshcore_hub/common/database.py b/src/meshcore_hub/common/database.py index 0ad7dff..02ebfd3 100644 --- a/src/meshcore_hub/common/database.py +++ b/src/meshcore_hub/common/database.py @@ -69,12 +69,19 @@ def create_database_engine( if database_url.startswith("sqlite"): connect_args["check_same_thread"] = False - # Scope Postgres connections to the configured schema via search_path. This keeps - # the models schema-agnostic (no hardcoded schema=) so the same code serves SQLite, + # Scope Postgres connections to the configured schema via search_path and pin + # the session timezone to UTC so func.date() truncates on the UTC + # day boundary — matching SQLite, which stores UTC text. This keeps the models + # schema-agnostic (no hardcoded schema=) so the same code serves SQLite, # single-instance Postgres, and multiple schema-isolated instances on one cluster. resolved_schema = _resolve_pg_schema(database_url, schema) - if resolved_schema: - connect_args["options"] = f"-csearch_path={resolved_schema}" + is_postgres = database_url.startswith(("postgresql", "postgres")) + if is_postgres: + options_parts: list[str] = [] + if resolved_schema: + options_parts.append(f"-csearch_path={resolved_schema}") + options_parts.append("-ctimezone=UTC") + connect_args["options"] = " ".join(options_parts) # Size the pool above the default Starlette threadpool (~40 threads) so # concurrent request handlers don't block waiting for a connection. Applies @@ -194,11 +201,17 @@ class DatabaseManager: async_url = _to_async_url(self.database_url) async_connect_args: dict[str, Any] = {} - # asyncpg sets search_path via server_settings (not the libpq -c options - # string the sync psycopg2 engine uses). self._schema is already resolved - # (explicit arg or DATABASE_SCHEMA env) and None for SQLite. - if self._schema: - async_connect_args["server_settings"] = {"search_path": self._schema} + # asyncpg sets search_path and timezone via server_settings (not the libpq + # -c options string the sync psycopg2 engine uses). self._schema is already + # resolved (explicit arg or DATABASE_SCHEMA env) and None for SQLite. + # Timezone is pinned to UTC unconditionally for Postgres so day boundaries + # match SQLite's UTC-text truncation. + is_postgres = self.database_url.startswith(("postgresql", "postgres")) + if is_postgres: + server_settings: dict[str, str] = {"timezone": "UTC"} + if self._schema: + server_settings["search_path"] = self._schema + async_connect_args["server_settings"] = server_settings self._async_engine = create_async_engine( async_url, echo=self._echo, connect_args=async_connect_args ) diff --git a/tests/test_api/conftest.py b/tests/test_api/conftest.py index 0546290..0e2b9e4 100644 --- a/tests/test_api/conftest.py +++ b/tests/test_api/conftest.py @@ -4,10 +4,12 @@ import os import tempfile from contextlib import contextmanager from datetime import datetime, timezone +from typing import Generator import pytest from fastapi.testclient import TestClient -from sqlalchemy import create_engine, event as sa_event +from sqlalchemy import create_engine, event as sa_event, text +from sqlalchemy.engine.url import make_url from sqlalchemy.orm import sessionmaker from meshcore_hub.api.app import create_app @@ -16,7 +18,7 @@ from meshcore_hub.api.dependencies import ( get_mqtt_client, get_db_manager, ) -from meshcore_hub.common.database import DatabaseManager +from meshcore_hub.common.database import DatabaseManager, create_database_engine from meshcore_hub.common.models import ( Advertisement, Base, @@ -46,24 +48,110 @@ def test_db_path(): @pytest.fixture(scope="session") -def api_db_engine(test_db_path): - """Session-scoped SQLite engine. Schema is built once per pytest session. +def db_backend() -> str: + """Active test database backend (``sqlite`` or ``postgres``). - Previously this was function-scoped and rebuilt ~15 tables for every test, - costing ~0.2s/test. Promoting it to session scope eliminates that. Per-test - isolation is handled by truncation in ``api_db_session``. + Controlled by ``TEST_DATABASE_BACKEND`` env var (default: ``sqlite``). + When ``postgres``, ``TEST_POSTGRES_URL`` must also be set. """ - db_url = f"sqlite:///{test_db_path}" - engine = create_engine( - db_url, - connect_args={"check_same_thread": False}, - ) + backend = os.environ.get("TEST_DATABASE_BACKEND", "sqlite").lower() + if backend not in ("sqlite", "postgres"): + raise ValueError( + f"TEST_DATABASE_BACKEND must be 'sqlite' or 'postgres', got: {backend}" + ) + return backend - @sa_event.listens_for(engine, "connect") - def set_sqlite_pragma(dbapi_connection: object, connection_record: object) -> None: - cursor = dbapi_connection.cursor() # type: ignore[attr-defined] - cursor.execute("PRAGMA foreign_keys=ON") - cursor.close() + +@pytest.fixture(scope="session") +def db_url(db_backend: str, test_db_path: str, request) -> Generator[str, None, None]: + """Database URL for the active backend. + + For Postgres, each pytest-xdist worker gets its own database (e.g. + ``test_gw0``) to avoid truncation races between parallel workers. + """ + if db_backend == "postgres": + env_url = os.environ.get("TEST_POSTGRES_URL") + if not env_url: + pytest.skip( + "TEST_DATABASE_BACKEND=postgres but TEST_POSTGRES_URL is not set; " + "e.g. TEST_POSTGRES_URL=postgresql+psycopg2://postgres:postgres@localhost:55432/test" + ) + assert env_url is not None + + worker_id = "master" + if hasattr(request.config, "workerinput"): + worker_id = request.config.workerinput["workerid"] + + base_url = make_url(env_url) + worker_db = f"{base_url.database}_{worker_id}" + worker_url = base_url.set(database=worker_db).render_as_string( + hide_password=False + ) + + admin_url = base_url.set(database="postgres") + admin_engine = create_engine( + admin_url.render_as_string(hide_password=False), + isolation_level="AUTOCOMMIT", + ) + try: + with admin_engine.connect() as conn: + exists = conn.execute( + text("SELECT 1 FROM pg_database WHERE datname = :name"), + {"name": worker_db}, + ).scalar() + if not exists: + conn.execute(text(f'CREATE DATABASE "{worker_db}"')) + finally: + admin_engine.dispose() + + yield worker_url + + admin_engine = create_engine( + admin_url.render_as_string(hide_password=False), + isolation_level="AUTOCOMMIT", + ) + try: + with admin_engine.connect() as conn: + conn.execute( + text( + "SELECT pg_terminate_backend(pid) " + "FROM pg_stat_activity " + "WHERE datname = :name AND pid <> pg_backend_pid()" + ), + {"name": worker_db}, + ) + conn.execute(text(f'DROP DATABASE IF EXISTS "{worker_db}"')) + finally: + admin_engine.dispose() + else: + yield f"sqlite:///{test_db_path}" + + +@pytest.fixture(scope="session") +def api_db_engine(db_url: str, db_backend: str): + """Session-scoped database engine. Schema is built once per pytest session. + + For Postgres, uses the production ``create_database_engine`` factory so the + test exercises the same ``connect_args`` (including ``-ctimezone=UTC``). + Each xdist worker has its own database (see ``db_url``), so no locking + is needed. + """ + if db_backend == "postgres": + engine = create_database_engine(db_url) + Base.metadata.drop_all(engine) + else: + engine = create_engine( + db_url, + connect_args={"check_same_thread": False}, + ) + + @sa_event.listens_for(engine, "connect") + def set_sqlite_pragma( + dbapi_connection: object, connection_record: object + ) -> None: + cursor = dbapi_connection.cursor() # type: ignore[attr-defined] + cursor.execute("PRAGMA foreign_keys=ON") + cursor.close() Base.metadata.create_all(engine) yield engine @@ -168,7 +256,7 @@ def _wire_overrides(app, api_db_engine, mock_mqtt, mock_db_manager) -> None: @pytest.fixture(scope="module") -def app_no_auth(test_db_path, api_db_engine, mock_mqtt, mock_db_manager): +def app_no_auth(db_url, api_db_engine, mock_mqtt, mock_db_manager): """Module-scoped FastAPI app with no authentication. Built once per test module; ``create_app`` is the second-most expensive @@ -176,7 +264,6 @@ def app_no_auth(test_db_path, api_db_engine, mock_mqtt, mock_db_manager): ``_isolate_db_global`` autouse fixture handles the global ``_db_manager`` so this fixture doesn't need a ``with patch(...)`` context. """ - db_url = f"sqlite:///{test_db_path}" app = create_app( database_url=db_url, read_key=None, @@ -187,9 +274,8 @@ def app_no_auth(test_db_path, api_db_engine, mock_mqtt, mock_db_manager): @pytest.fixture(scope="module") -def app_with_auth(test_db_path, api_db_engine, mock_mqtt, mock_db_manager): +def app_with_auth(db_url, api_db_engine, mock_mqtt, mock_db_manager): """Module-scoped FastAPI app with authentication enabled.""" - db_url = f"sqlite:///{test_db_path}" app = create_app( database_url=db_url, read_key="test-read-key", diff --git a/tests/test_api/test_dashboard.py b/tests/test_api/test_dashboard.py index bb4b411..02e58cb 100644 --- a/tests/test_api/test_dashboard.py +++ b/tests/test_api/test_dashboard.py @@ -1,10 +1,11 @@ """Tests for dashboard API routes.""" -from datetime import datetime, timedelta, timezone +from datetime import date, datetime, timedelta, timezone from unittest.mock import patch import pytest +from meshcore_hub.api.routes.dashboard import _date_bucket_key from meshcore_hub.common.models import ( Advertisement, Message, @@ -15,6 +16,31 @@ from meshcore_hub.common.models import ( from meshcore_hub.common.models import UserProfile +class TestDateBucketKey: + """Unit tests for the _date_bucket_key dialect-neutral normalization helper.""" + + def test_str_passthrough(self) -> None: + """SQLite returns str — pass through unchanged.""" + assert _date_bucket_key("2026-06-15") == "2026-06-15" + + def test_date_object_normalized(self) -> None: + """Postgres returns datetime.date — coerce to %Y-%m-%d string.""" + assert _date_bucket_key(date(2026, 1, 5)) == "2026-01-05" + + def test_datetime_object_normalized(self) -> None: + """Postgres may return datetime.datetime — coerce to %Y-%m-%d string.""" + dt = datetime(2026, 6, 15, 14, 30, 0, tzinfo=timezone.utc) + assert _date_bucket_key(dt) == "2026-06-15" + + def test_none_passthrough(self) -> None: + """None passes through unchanged (no collision with string lookups).""" + assert _date_bucket_key(None) is None + + def test_zero_padded_date(self) -> None: + """Single-digit months/days are zero-padded.""" + assert _date_bucket_key(date(2026, 1, 5)) == "2026-01-05" + + class TestDashboardStats: """Tests for GET /dashboard/stats endpoint.""" @@ -231,9 +257,16 @@ class TestDashboardActivity: response = client_no_auth.get("/api/v1/dashboard/activity") assert response.status_code == 200 data = response.json() - # At least one day should have a count > 0 total_count = sum(point["count"] for point in data["data"]) assert total_count >= 1 + # The seeded advertisement was yesterday — its bucket must be non-zero. + # This catches the Postgres flatline bug where func.date() returns a + # date object that never matches the string key. + yesterday_str = (datetime.now(timezone.utc) - timedelta(days=1)).strftime( + "%Y-%m-%d" + ) + yesterday_point = next(p for p in data["data"] if p["date"] == yesterday_str) + assert yesterday_point["count"] >= 1 class TestMessageActivity: @@ -291,9 +324,14 @@ class TestMessageActivity: response = client_no_auth.get("/api/v1/dashboard/message-activity") assert response.status_code == 200 data = response.json() - # At least one day should have a count > 0 total_count = sum(point["count"] for point in data["data"]) assert total_count >= 1 + # The seeded message was yesterday — its bucket must be non-zero. + yesterday_str = (datetime.now(timezone.utc) - timedelta(days=1)).strftime( + "%Y-%m-%d" + ) + yesterday_point = next(p for p in data["data"] if p["date"] == yesterday_str) + assert yesterday_point["count"] >= 1 class TestNodeCountHistory: @@ -356,6 +394,16 @@ class TestNodeCountHistory: # At least one day should have a count > 0 (cumulative) # The last day should have count >= 1 assert data["data"][-1]["count"] >= 1 + # The node was created yesterday — the cumulative count should step + # up at yesterday's bucket and stay >= 1 for all subsequent days. + yesterday_str = (datetime.now(timezone.utc) - timedelta(days=1)).strftime( + "%Y-%m-%d" + ) + yesterday_idx = next( + i for i, p in enumerate(data["data"]) if p["date"] == yesterday_str + ) + assert data["data"][yesterday_idx]["count"] >= 1 + assert all(p["count"] >= 1 for p in data["data"][yesterday_idx:]) def test_node_count_is_cumulative_with_baseline( self, client_no_auth, api_db_session @@ -770,3 +818,88 @@ class TestDashboardChannelVisibility: assert str(pub_idx) in data["channel_message_counts"] assert str(mem_idx) in data["channel_message_counts"] assert str(adm_idx) not in data["channel_message_counts"] + + +class TestDashboardDateBucketRegression: + """Regression tests for the Postgres date-bucket flatline bug. + + These tests seed data at deterministic UTC timestamps and assert the + specific date buckets have non-zero counts. On SQLite (where + func.date() returns str) these always passed. On Postgres (where + func.date() returns date) these would have failed before the + _date_bucket_key fix, because the dict lookup by string key would + miss the date-object key. + + Run against Postgres with:: + + TEST_DATABASE_BACKEND=postgres \\ + TEST_POSTGRES_URL=postgresql+psycopg2://postgres:postgres@localhost:55432/test \\ + pytest tests/test_api/test_dashboard.py -k Regression + """ + + def test_activity_nonzero_on_seeded_day(self, client_no_auth, api_db_session): + """Activity chart shows non-zero count for the seeded day.""" + two_days_ago = (datetime.now(timezone.utc) - timedelta(days=2)).replace( + hour=12, minute=0, second=0, microsecond=0 + ) + api_db_session.add( + Advertisement( + public_key="a" * 64, + name="RegressionNode", + adv_type="REPEATER", + route_type="flood", + received_at=two_days_ago, + ) + ) + api_db_session.commit() + + data = client_no_auth.get("/api/v1/dashboard/activity").json() + seeded_str = two_days_ago.strftime("%Y-%m-%d") + seeded_point = next(p for p in data["data"] if p["date"] == seeded_str) + assert seeded_point["count"] == 1 + + def test_message_activity_nonzero_on_seeded_day( + self, client_no_auth, api_db_session + ): + """Message activity chart shows non-zero count for the seeded day.""" + two_days_ago = (datetime.now(timezone.utc) - timedelta(days=2)).replace( + hour=12, minute=0, second=0, microsecond=0 + ) + api_db_session.add( + Message( + message_type="direct", + pubkey_prefix="abc123", + text="regression test", + received_at=two_days_ago, + ) + ) + api_db_session.commit() + + data = client_no_auth.get("/api/v1/dashboard/message-activity").json() + seeded_str = two_days_ago.strftime("%Y-%m-%d") + seeded_point = next(p for p in data["data"] if p["date"] == seeded_str) + assert seeded_point["count"] == 1 + + def test_node_count_steps_up_on_seeded_day(self, client_no_auth, api_db_session): + """Node count chart steps up on the seeded creation day.""" + two_days_ago = (datetime.now(timezone.utc) - timedelta(days=2)).replace( + hour=12, minute=0, second=0, microsecond=0 + ) + api_db_session.add( + Node( + public_key="b" * 64, + name="RegressionNode", + created_at=two_days_ago, + ) + ) + api_db_session.commit() + + data = client_no_auth.get("/api/v1/dashboard/node-count").json() + seeded_str = two_days_ago.strftime("%Y-%m-%d") + seeded_idx = next( + i for i, p in enumerate(data["data"]) if p["date"] == seeded_str + ) + # Before the seeded day: 0 nodes. On/after: 1. + assert data["data"][seeded_idx]["count"] == 1 + if seeded_idx > 0: + assert data["data"][seeded_idx - 1]["count"] == 0 diff --git a/tests/test_api/test_user_profiles.py b/tests/test_api/test_user_profiles.py index 7cb4c90..93e4153 100644 --- a/tests/test_api/test_user_profiles.py +++ b/tests/test_api/test_user_profiles.py @@ -95,13 +95,23 @@ class TestListProfiles: api_db_session.add(adoption) api_db_session.commit() - api_db_session.execute(text("PRAGMA foreign_keys=OFF")) + # Temporarily disable FK enforcement to simulate an orphaned adoption + # (node deleted while the adoption record persists). SQLite uses + # PRAGMA; Postgres uses session_replication_role = replica. + dialect = api_db_session.bind.dialect.name # type: ignore[union-attr] + if dialect == "postgresql": + api_db_session.execute(text("SET session_replication_role = replica")) + else: + api_db_session.execute(text("PRAGMA foreign_keys=OFF")) api_db_session.execute( text("DELETE FROM nodes WHERE id = :id"), {"id": sample_node.id}, ) api_db_session.commit() - api_db_session.execute(text("PRAGMA foreign_keys=ON")) + if dialect == "postgresql": + api_db_session.execute(text("SET session_replication_role = DEFAULT")) + else: + api_db_session.execute(text("PRAGMA foreign_keys=ON")) response = client_no_auth.get( "/api/v1/user/profiles", diff --git a/tests/test_common/test_database.py b/tests/test_common/test_database.py index 51dc9a0..05afee4 100644 --- a/tests/test_common/test_database.py +++ b/tests/test_common/test_database.py @@ -1,11 +1,13 @@ """Tests for database engine configuration.""" from pathlib import Path +from unittest.mock import patch import pytest from sqlalchemy import text from meshcore_hub.common.database import ( + DatabaseManager, _resolve_pg_schema, _to_async_url, create_database_engine, @@ -78,3 +80,72 @@ class TestSchemaResolution: def test_none_when_no_env(self, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.delenv("DATABASE_SCHEMA", raising=False) assert _resolve_pg_schema("postgresql://u@h/db", None) is None + + +class TestPostgresSessionTimezone: + """Verify Postgres connections are pinned to UTC at the engine level. + + func.date() truncates on the session timezone's day boundary. + The collector writes UTC, so the session must be UTC for day buckets to + match SQLite's UTC-text truncation. + """ + + def test_sync_engine_sets_timezone_utc_without_schema( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Postgres engine without a schema still pins timezone=UTC.""" + monkeypatch.delenv("DATABASE_SCHEMA", raising=False) + with patch("meshcore_hub.common.database.create_engine") as mock_create: + create_database_engine("postgresql://u:p@h/db") + _, kwargs = mock_create.call_args + assert kwargs["connect_args"]["options"] == "-ctimezone=UTC" + + def test_sync_engine_timezone_utc_with_schema(self) -> None: + """Postgres engine with a schema sets both search_path and timezone.""" + with patch("meshcore_hub.common.database.create_engine") as mock_create: + create_database_engine("postgresql://u:p@h/db", schema="meshcorehub") + _, kwargs = mock_create.call_args + options = kwargs["connect_args"]["options"] + assert "-csearch_path=meshcorehub" in options + assert "-ctimezone=UTC" in options + + def test_sqlite_engine_has_no_timezone_options(self, tmp_path: Path) -> None: + """SQLite engines must not set timezone options.""" + engine = create_database_engine(f"sqlite:///{tmp_path / 'x.db'}") + try: + assert "options" not in engine.url.query + finally: + engine.dispose() + + def test_async_engine_sets_server_settings_timezone_utc( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """asyncpg engine gets server_settings with timezone=UTC.""" + monkeypatch.delenv("DATABASE_SCHEMA", raising=False) + manager = DatabaseManager.__new__(DatabaseManager) + manager.database_url = "postgresql://u:p@h/db" + manager._echo = False + manager._schema = None + manager._async_engine = None + manager._async_session_factory = None + + with patch("meshcore_hub.common.database.create_async_engine") as mock_async: + manager._ensure_async_engine() + _, kwargs = mock_async.call_args + assert kwargs["connect_args"]["server_settings"] == {"timezone": "UTC"} + + def test_async_engine_sets_server_settings_with_schema(self) -> None: + """asyncpg engine with schema gets both search_path and timezone.""" + manager = DatabaseManager.__new__(DatabaseManager) + manager.database_url = "postgresql://u:p@h/db" + manager._echo = False + manager._schema = "meshcorehub" + manager._async_engine = None + manager._async_session_factory = None + + with patch("meshcore_hub.common.database.create_async_engine") as mock_async: + manager._ensure_async_engine() + _, kwargs = mock_async.call_args + server_settings = kwargs["connect_args"]["server_settings"] + assert server_settings["timezone"] == "UTC" + assert server_settings["search_path"] == "meshcorehub"