Replaces the httpx-based smoke tests in tests/e2e/ with a browser E2E suite
under e2e/, running against a self-contained throwaway stack that never
touches the local development database.
Stack & data isolation (e2e/docker-compose.test.yml):
- Own ephemeral Postgres 17 (schema via the `migrate` service / Alembic),
distinct project name + named volumes, no host DB port, no \${VAR}
interpolation. `make e2e-down` destroys everything.
- OIDC enabled with a known session secret; WEB_AUTO_REFRESH_SECONDS=2 so
polling is assertable; CONTENT_HOME mounts a test markdown page.
- Deterministic seeder (e2e/seed_data.py) run via global setup inside the
collector container: nodes/observers with `area` tags, adverts, messages
on the public (17) + a custom channel, raw packets + path hops keyed to
node prefixes, a route + health/history, profiles + adoptions, and
event_observers rows so the observer filter/badges resolve.
Auth & tests:
- Forged signed `meshcore-session` cookies (e2e/mint_session.py,
itsdangerous) for admin/member identities -> storageState; no real IdP
needed. 31 specs across 12 files cover global nav/theme, profile
menu+edit, home hero, dashboard widgets, list filters/auto-refresh/row
actions, observer toggles, the path-node overlay, map filters +
show-labels, members, markdown pages, and routes add/edit/delete
(persistence, validation, confirm dialog). workers:1 / fullyParallel:false
against the single shared backend; targeted data-testids added to React
components for stable selectors.
Fixes surfaced while running the suite on fresh Postgres:
- Migration 5e3b712ccf10 aborted on Postgres: its route-health backfill
queries the live Route model (which now has max_path_length) before that
column exists. The swallowed error left the transaction aborted, killing
the subsequent alembic_version stamp. Wrapped the backfill in a SAVEPOINT
so a failure rolls back cleanly without blocking the migration.
- Restored the full ABUSE_* env set required by the MQTT broker, and set
the web service's API_KEY to the admin key (admin writes go through the
proxy as a Bearer token).
31/31 passing; tsc (frontend+e2e), pytest (1460), and pre-commit green.
FastAPI 0.137.0 refactored include_router to keep included routers as
nested objects rather than flattening their routes into app.routes, so
test_app_factory's `{route.path for route in app.routes}` no longer found
the /metrics route (the endpoint still serves; only this introspection
broke). FastAPI now treats router.routes as an internal implementation
detail.
Switch the metrics route checks to the public OpenAPI schema
(app.openapi()["paths"]), which is stable across versions and resolves
router prefixes correctly, and drop the <0.137.0 pin that was blocking
the upgrade.
Verified: app factory tests pass on both 0.136.3 and 0.137.2; full
tests/test_api suite (462 tests) passes on 0.137.2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dashboard charts (activity, message-activity, node-count) rendered as
flat zeros on Postgres because func.date() returns a str on SQLite but
a datetime.date on Postgres — the dict lookup by string key always
missed. Fixed with a dialect-neutral _date_bucket_key() helper and
pinned the Postgres session timezone to UTC at the engine level.
Also adds dual-backend test infrastructure (TEST_DATABASE_BACKEND env
var), per-worker Postgres databases for pytest-xdist isolation, and
strengthened regression tests asserting non-zero date buckets.
FastAPI 0.137.0 regressed include_router: endpoints registered via
app.include_router no longer appear in app.routes (they still serve, but
introspection breaks). tests/test_api/test_app_factory.py asserts on
app.routes, so CI (which pip-installs the latest fastapi) failed on the
/metrics route check while local pinned 0.136.3 passed. Constrain fastapi
to <0.137.0 until a fixed release is available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace subprocess-based packet decoding with native Python meshcoredecoder>=0.3.2.
The decoder is now always enabled (no toggle), removing ENABLED/COMMAND/TIMEOUT
config vars. Adds _enrich_payload_decoded() to compensate for payload classes that
lack to_dict() overrides in the library. Removes Node.js/npm from Dockerfile and
deletes the patches/ directory. Adds parenthesized exception rule to AGENTS.md.
Remove the meshcore_interface component in favor of external
meshcore-packet-capture for data ingestion. Rename receiver_node_id
to observer_node_id across all models, schemas, handlers, and API
routes. Add Alembic migration for the column/table renames. Fix
frontend JS property name mismatch that prevented the Receiver column
from displaying observer data.
Pin starlette<1.0.0 to avoid TemplateResponse breaking change, update
the TemplateResponse call to new-style request-first signature, and
bump Python version to 3.14 across Dockerfile, pyproject.toml, and
pre-commit hooks.
Update path hash handling to accept variable-length hex-encoded hashes
(e.g. "4a" for single-byte, "b3fa" for multibyte) instead of requiring
exactly 2-character hashes. Bump meshcore dependency to >=2.3.0.
- Update normalizer to accept even-length hex strings >= 2 chars
- Update schemas and model docstrings for variable-length hashes
- Add tests for multibyte and mixed-length path hash round-trips
- Fix web test flakiness from local .env datetime locale leaking
Add /metrics endpoint with Prometheus gauges for nodes, messages,
advertisements, telemetry, trace paths, events, and members. Include
per-node last_seen timestamps for alerting. Add Alertmanager service
to Docker Compose metrics profile with default blackhole receiver.
Add NodeNotSeen alert rule (48h threshold). Add 1h time window to
all windowed metrics alongside existing 24h/7d/30d windows.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Allows adding static content pages (About, FAQ, etc.) as markdown files
with YAML frontmatter. Pages are stored in PAGES_HOME directory (default:
./pages), automatically appear in navigation menu, and are included in
the sitemap.
- Add PageLoader class to parse markdown with frontmatter
- Add /pages/{slug} route for rendering custom pages
- Add PAGES_HOME config setting to WebSettings
- Add prose CSS styles for markdown content
- Add pages to navigation and sitemap
- Update docker-compose.yml with pages volume mount
- Add comprehensive tests for PageLoader and routes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add CONTACT_CLEANUP_ENABLED and CONTACT_CLEANUP_DAYS settings
- Implement remove_contact and schedule_remove_contact on device classes
- During contact sync, remove stale contacts from companion node
- Stale contacts (not advertised for > N days) not published to MQTT
- Update Python version to 3.13 across project config
- Remove brittle config tests that assumed default env values
- Update license from MIT to GPL-3.0-or-later in pyproject.toml
- Update project URLs from meshcore-dev to ipnet-mesh organization
- Add explicit GPL-3.0 license statement to README
- Fix AGENTS.md venv directory reference (.venv vs venv)
- Remove undocumented NETWORK_LOCATION from README
- Fix stats endpoint path in README (/api/v1/dashboard/stats)
- Clarify seed and data directory descriptions in project structure
- Replace JSON seed files with YAML format for better readability
- Auto-detect YAML primitive types (number, boolean, string) from values
- Add automatic seed import on collector startup
- Split lat/lon into separate tags instead of combined coordinate string
- Add PyYAML dependency and types-PyYAML for type checking
- Update example/seed and contrib/seed/ipnet with clean YAML format
- Update tests to verify YAML primitive type detection
- Update .flake8 and pre-commit config to properly use flake8 config
- Add B008 to ignored errors (FastAPI Depends pattern)
- Add E402 to ignored errors (intentional module-level imports)
- Remove unused imports from test files and source files
- Fix f-strings without placeholders
- Add type annotations to inner async functions
- Fix SQLAlchemy execute() to use text() wrapper
- Add type: ignore comments for alembic.command imports
- Exclude alembic/ directory from mypy in pre-commit
- Update mypy overrides for test files to not require type annotations
- Fix type annotations for params dicts in web routes
- Fix generator return type in test fixtures
CLI fixes:
- Add --mqtt-username and --mqtt-password options to receiver/sender
shortcut commands so they work with authenticated MQTT brokers
- These options read from MQTT_USERNAME/MQTT_PASSWORD env vars
Device integration:
- Integrate with meshcore>=2.2.0 library for actual serial device support
- Implement async-to-sync bridge for meshcore's async API
- Add proper event subscription mapping between meshcore and hub events
- Add meshcore>=2.2.0 to dependencies in pyproject.toml
- Add load_dotenv() call at module import to load .env before Click
processes envvar parameters
- Add python-dotenv>=1.0.0 as explicit dependency in pyproject.toml
Click's envvar only reads from OS environment, not .env files.
By loading dotenv early, all CLI commands now respect .env configuration.
This commit establishes the complete foundation for the MeshCore Hub project:
- Project setup with pyproject.toml (Python 3.11+, all dependencies)
- Development tools: black, flake8, mypy, pytest configuration
- Pre-commit hooks for code quality
- Package structure with all components (interface, collector, api, web)
Common package includes:
- Pydantic settings for all component configurations
- SQLAlchemy models for nodes, messages, advertisements, traces, telemetry
- Pydantic schemas for events, API requests/responses, commands
- MQTT client utilities with topic builder
- Logging configuration
Database infrastructure:
- Alembic setup with initial migration for all tables
- Database manager with session handling
CLI entry point:
- Click-based CLI with subcommands for all components
- Database migration commands (upgrade, downgrade, revision)
Tests:
- Basic test suite for config and models
- pytest fixtures for in-memory database testing
Louis King