meshcore-hub/AGENTS.md

AGENTS.md - AI Coding Assistant Guidelines

Critical Rules (MUST follow)

• Always use parenthesized exception tuples — except (ValueError, TypeError): not except ValueError, TypeError:. The comma form is Python 2 syntax and fails at import time in Python 3. The most common error that passes visual review but breaks the app.
• To run tests and get results, use:

  pytest --no-cov 2>&1 | grep -iE "passed|failed" | tail -3
  

  --no-cov skips coverage for speed; the pipe surfaces only the pass/fail summary.
• Use Python (version in .python-version); activate a venv in .venv before running pytest, pre-commit, or alembic locally.
• All other operations run inside the compose stack — never invoke meshcore-hub or npm directly on the host; build/run/exec via docker compose (see Development).
• Never git push without explicit confirmation — staging and committing discrete changes is fine.
• Never build the Docker images or run make build / make up — the user builds manually to test. Stop after code changes + tests + pre-commit pass.
• Always generate random Alembic revision IDs — use python -c "import secrets; print(secrets.token_hex(6))" or let alembic revision auto-generate. Never hand-pick sequential or guessable IDs like a1b2c3d4e5f6 — they collide with existing migrations and cause cycle errors at upgrade time.
• Before committing: run targeted pytest --no-cov tests/test_<component>/ then pre-commit run --all-files.

Setup

ls ./.venv || python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pre-commit install

This venv is only for local testing, linting, and migration authoring. Frontend assets and runtime deps build into the Docker image — there is no local npm step.

Development

# Build / start / stop the stack (core = collector + api + web + migrate)
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core build
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core up -d
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core down

# Run a command inside a running service
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core exec <service> <command>

# Shorthands (Makefile, mqtt+core profiles): make build | make up | make down | make logs

Tests & Quality

Coverage is opt-in; add --cov=meshcore_hub (or make test-cov) when you want it. The dev loop defaults to no coverage and parallel across CPU cores.

# Canonical: run tests in parallel, no coverage, surface the pass/fail summary
pytest -nauto --no-cov 2>&1 | grep -iE "passed|failed" | tail -3

# Makefile shorthands
make test        # pytest -nauto --no-cov (parallel dev loop)
make test-cov    # full run with coverage report
make test-unit   # parallel, fast unit suites only (skips e2e)

# Targeted by component (run only what you changed)
pytest --no-cov tests/test_web/        # templates, static JS, web routes
pytest --no-cov tests/test_api/        # API changes
pytest --no-cov tests/test_collector/  # collector changes
pytest --no-cov tests/test_common/     # common models/schemas/config

# Full suite only if changes span multiple components
pytest --no-cov

# Quality checks
pre-commit run --all-files

Database & Ops

The default backend is SQLite (zero-config, file at ${DATA_HOME}/collector/meshcore.db). PostgreSQL is also supported via DATABASE_BACKEND=postgres — see docs/database.md for the full backend reference, production provisioning, and schema-per-instance setup. Migrations are backend-agnostic; the commands below work for both.

# --- LOCAL (venv): sync the volume DB to ./meshcore.db, then author a migration
# Volume name is ${COMPOSE_PROJECT_NAME:-hub}_data (default: hub_data)
# (SQLite only — for Postgres, point the migration env at the cluster directly)
docker run -it --rm -v hub_data:/data -v "$PWD":/pwd ubuntu cp /data/collector/meshcore.db /pwd/meshcore.db
meshcore-hub db revision --autogenerate -m "description"

# --- CONTAINER: apply migrations (the DB lives in the data volume)
# Migrations auto-apply on `up` via the migrate service. Manual one-off:
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core run --rm migrate db upgrade

# --- CONTAINER: seed node tags from SEED_HOME (NOT automatic)
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile seed run --rm seed

# --- CONTAINER: data retention / node cleanup (exec into the running collector)
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core exec collector meshcore-hub collector cleanup --retention-days 30 --dry-run
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile core exec collector meshcore-hub collector cleanup --retention-days 30

Conventions

Cache invalidation on writes

Every mutation handler (POST/PUT/DELETE) on a user/admin-mutable entity MUST call the matching invalidate_* helper from meshcore_hub.api.cache_invalidation after session.commit() succeeds, so the UI reflects the change on the next page load instead of waiting for the Redis TTL. The helper is a no-op when Redis is disabled and swallows backend errors, so it's always safe to call.

The HTTP-layer cache policy on /api/v1/* GETs is private, no-cache (i.e. must-revalidate) precisely so this works: the browser always sends If-None-Match on navigation, the server answers 304 when Redis is warm and unchanged (cheap — no body) or 200 after an invalidation. Do NOT change this back to max-age>0 — server-side cache invalidation cannot reach the browser's HTTP cache, so any freshness window would let stale responses survive a mutation until expiry.

from meshcore_hub.api.cache_invalidation import invalidate_channels

@router.put("/{channel_id}")
def update_channel(__: RequireAdmin, session: DbSession, channel_id: str,
                   body: ChannelUpdate, request: Request) -> ChannelRead:
    # ... mutate ...
    session.commit()
    session.refresh(channel)
    invalidate_channels(request)   # after commit, before return
    return _channel_to_read(channel)

Mapping (see api/cache_invalidation.py for the canonical prefix knowledge):

Mutation	Helper(s)
POST/PUT/DELETE /channels	invalidate_channels
POST/PUT/DELETE /routes	invalidate_routes (covers list, detail, history)
PUT /user/profile/{id}	invalidate_profiles + invalidate_dashboard
POST/PUT/DELETE /nodes/{pk}/tags	invalidate_nodes + invalidate_messages + invalidate_advertisements + invalidate_dashboard (tags drive names/filters across these)
POST/DELETE /adoptions	invalidate_nodes + invalidate_profiles + invalidate_advertisements + invalidate_dashboard (adopted_by embedded across these)

When adding a new @cached read endpoint, decide whether its key namespace belongs in an existing invalidate helper, and add a test in tests/test_api/test_cache.py::TestMutationInvalidationIntegration. Cache keys split across two formats (endpoint-name keys like nodes: vs URL-path keys like /api/v1/channels:) — the helper module encapsulates that, don't hand-roll prefixes.

# Imports: stdlib, third-party, local
import os
from datetime import datetime
from typing import Optional

from fastapi import FastAPI, Depends
from pydantic import BaseModel
from sqlalchemy import select

from meshcore_hub.common.config import Settings
from meshcore_hub.common.models import Node

# Pydantic model
from pydantic import BaseModel, Field
from datetime import datetime
from typing import Optional

class NodeRead(BaseModel):
    """Schema for reading node data from API."""

    id: str
    public_key: str = Field(..., min_length=64, max_length=64)
    name: Optional[str] = None
    adv_type: Optional[str] = None
    last_seen: Optional[datetime] = None

    model_config = {"from_attributes": True}

# SQLAlchemy model
from sqlalchemy import String, DateTime, Text
from sqlalchemy.orm import Mapped, mapped_column, relationship
from typing import Optional

from meshcore_hub.common.models.base import Base, TimestampMixin, UUIDMixin

class Node(Base, UUIDMixin, TimestampMixin):
    __tablename__ = "nodes"

    public_key: Mapped[str] = mapped_column(String(64), unique=True, index=True)
    name: Mapped[str | None] = mapped_column(String(255), nullable=True)
    adv_type: Mapped[str | None] = mapped_column(String(20), nullable=True)
    last_seen: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)

    tags: Mapped[list["NodeTag"]] = relationship(back_populates="node", cascade="all, delete-orphan")


class UserProfile(Base, UUIDMixin, TimestampMixin):
    """UserProfile model for authenticated OIDC users."""
    __tablename__ = "user_profiles"

    user_id: Mapped[str] = mapped_column(String(255), unique=True, nullable=False, index=True)
    name: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
    callsign: Mapped[Optional[str]] = mapped_column(String(20), nullable=True)
    roles: Mapped[Optional[str]] = mapped_column(Text, nullable=True)

# FastAPI route
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy.ext.asyncio import AsyncSession
from typing import Annotated

from meshcore_hub.api.dependencies import get_db, require_read
from meshcore_hub.common.schemas import NodeRead, NodeList

router = APIRouter(prefix="/nodes", tags=["nodes"])

@router.get("", response_model=NodeList)
async def list_nodes(
    db: Annotated[AsyncSession, Depends(get_db)],
    _: Annotated[None, Depends(require_read)],
    limit: int = Query(default=50, le=100),
    offset: int = Query(default=0, ge=0),
) -> NodeList:
    """List all nodes with pagination."""
    pass

# Click CLI command
import click

@click.group()
@click.pass_context
def cli(ctx: click.Context) -> None:
    """MeshCore Hub CLI."""
    ctx.ensure_object(dict)

@cli.command()
@click.option("--host", default="0.0.0.0", help="Bind host")
@click.option("--port", default=8000, type=int, help="Bind port")
@click.pass_context
def api(ctx: click.Context, host: str, port: int) -> None:
    """Start the API server."""
    import uvicorn
    from meshcore_hub.api.app import create_app

    app = create_app()
    uvicorn.run(app, host=host, port=port)

# Async lifespan
from contextlib import asynccontextmanager
from typing import AsyncGenerator
from fastapi import FastAPI

@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
    """Application lifespan handler."""
    await setup_database()
    await connect_mqtt()

    yield

    await disconnect_mqtt()
    await close_database()

# Error handling
from fastapi import HTTPException, status
import logging

logger = logging.getLogger(__name__)

raise HTTPException(
    status_code=status.HTTP_404_NOT_FOUND,
    detail=f"Node with public_key '{public_key}' not found"
)

try:
    result = await risky_operation()
except SomeException as e:
    logger.exception("Failed to perform operation: %s", e)
    raise

Test patterns

# Unit test
import pytest
from unittest.mock import AsyncMock, patch

@pytest.mark.asyncio
async def test_collector_handles_advertisement():
    """Test that collector handler processes advertisement events."""
    handler = AdvertisementHandler(db_session=AsyncMock())

    await handler.handle(event_data)

    handler.db_session.add.assert_called_once()
    node = handler.db_session.add.call_args[0][0]
    assert node.public_key == event_data["public_key"]

# Integration test
import pytest
from httpx import AsyncClient
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession

@pytest.fixture
async def db_session():
    """Create in-memory SQLite database for testing."""
    engine = create_async_engine("sqlite+aiosqlite:///:memory:")
    async with engine.begin() as conn:
        await conn.run_sync(Base.metadata.create_all)

    async with AsyncSession(engine) as session:
        yield session

@pytest.fixture
async def client(db_session):
    """Create test client with database session."""
    app = create_app()
    app.dependency_overrides[get_db] = lambda: db_session

    async with AsyncClient(app=app, base_url="http://test") as client:
        yield client

Where to find the rest

• README.md — project overview, deployment, MQTT topics, node tags, data retention, troubleshooting
• SCHEMAS.md — API/data schemas
• .env.example — all environment variables with defaults and comments
• docs/auth.md — OIDC authentication and roles
• docs/content.md — custom content (CONTENT_HOME)
• docs/i18n.md — translation reference
• docs/letsmesh.md — packet decoding and MQTT feed details
• docs/seeding.md — seed data
• docs/upgrading.md — upgrade notes
• docs/webhooks.md — webhook configuration and payloads